HTTPS flow decryption method based on non-perceptual differentiation original TCP connection technology

Abstract

The invention discloses an HTTPS (Hyper Text Transfer Protocol Secure) traffic decryption method based on a non-perceptual differentiation original TCP (Transmission Control Protocol) connection technology, which comprises the following steps of: relying on a user networking system, setting in an HTTPS domain name to be decrypted, and issuing DPI (Deep Packet Inspection) equipment backflow traffic; the forwarding module processes traffic, and the monitoring module establishes two types of HTTPS connections; identifying uplink flow, matching domain names, distinguishing whether decryption is carried out or not, respectively carrying out reinjection or differentiation processing, and completing interaction of two types of connection data; according to the method, proxy nodes and complex configuration are not needed, HTTPS session hijacking decryption can be realized by accessing normal internet traffic, and the method can be deployed at any position such as universities, enterprises, provincial network backbones and international nodes. According to the method, on the premise that established TCP connection is not interfered, no perceptual intervention exists in the TLS handshake stage, connection is differentiated, an original TCP session quintuple is reused to establish a bidirectional HTTPS channel which is not perceptual to the outside, and decryption and forwarding of encrypted data are completed.

Description

Technical Field

[0001] The present invention relates to the technical field of communications, and in particular to an HTTPS traffic decryption method based on a technology of non-perceptually differentiating original TCP connections. Background Art

[0002] While the currently used man-in-the-middle (MITM) decryption solution can decrypt HTTPS traffic, it exhibits several significant flaws in practical applications. Deployment requires the precise deployment of proxy nodes at the egress point of the compromised client. This means that the network architecture must configure a separate physical or virtual proxy entity for each client to be monitored. For example, in an enterprise network environment, traffic monitoring for hundreds of endpoints requires not only deploying proxy nodes at each device's egress gateway but also ensuring accurate TCP communication configuration between the proxy nodes and the clients. This requires clients to manually modify their system network settings, configure the proxy server's IP and port, and, in some scenarios, install and trust the proxy node's root certificate to avoid browser security warnings. These operations are technically demanding for the average user and are prone to configuration consistency issues in large-scale deployments. For example, proxy configuration interfaces differ across operating systems (Windows, macOS, and Linux), and conflicting priorities between browser and system proxy settings. This makes this solution difficult to implement in network environments with varying technical expertise, such as those in enterprises and schools, limiting its application to individual devices or small-scale labs.

[0003] In terms of usability, the complex pre-configuration process has become a major pain point. In traditional network access scenarios, users only need to configure DNS or gateways to access the network, but this solution requires users to actively participate in the connection process of the proxy node: first, they need to manually specify the proxy server address through the command line or graphical interface, secondly, they need to ensure that the port of the proxy node is not blocked by the firewall, and finally, they need to deal with the certificate trust issue during HTTPS connection. For non-technical users, configuration errors in any link will lead to traffic takeover failure, and troubleshooting requires multiple detections at the network layer, transport layer, and application layer, which greatly increases operation and maintenance costs. For example, in public Wi-Fi scenarios, it is obviously unrealistic to require each access user to configure the proxy node by themselves, which directly leads to the inability of this solution to be implemented in an open network environment.

[0004] From a network behavior perspective, the centralized forwarding mechanism of proxy nodes leads to highly homogenized traffic patterns. After all client traffic is forwarded through the proxy nodes, the source IP addresses of all requests received by the target server are all from the proxy node's single IP address. This unusual traffic aggregation is easily detected by the target server's anti-proxy detection system. For example, on e-commerce platforms or social media platforms, their risk control systems often trigger security alerts based on frequent access from the same IP address, quickly blacklisting the proxy node IP address and thus impacting normal traffic forwarding. Furthermore, the uniformity of source IP addresses can render network audits ineffective. When tracing the network behavior of a specific client, only the proxy node can be located, not the actual client. This can lead to the loss of critical information during cybersecurity incident investigations. More importantly, this proxy behavior can trigger intervention by ISPs (Internet Service Providers), such as using DPI (Deep Packet Inspection) technology to identify the proxy node's communication patterns and block traffic, rendering the entire decryption solution ineffective.

[0005] Further analysis reveals inherent flaws in the technical architecture: Proxy nodes, as intermediate hubs for traffic forwarding, essentially become single-point bottlenecks in network communications. As the number of clients increases or traffic loads rise, the proxy node's CPU power (used for SSL / TLS encryption and decryption), memory bandwidth (used for data caching), and network IO capabilities may become performance bottlenecks, leading to increased data forwarding delays and even packet loss. At the same time, the security of proxy nodes faces a dual challenge: they must ensure they are not infiltrated by attacks from the client or target server, while also preventing decrypted plaintext data from being illegally stolen within the node. In existing solutions, proxy nodes are typically exposed on the communication link between the client and the ISP. Their inherent security protection level directly determines the reliability of the entire decryption system. Complex network attacks (such as variants of man-in-the-middle attacks targeting SSL protocol vulnerabilities) can bypass the proxy node's security mechanisms, causing the decryption process to fail or data to be leaked. Summary of the Invention

[0006] The purpose of the present invention is to provide an HTTPS traffic decryption method based on the technology of non-perceptual differentiation of original TCP connection to solve the problems in the above-mentioned prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for decrypting HTTPS traffic based on the technology of non-perceptual differentiation of original TCP connections, the method comprising: S1. Relying on the existing user Internet access system, the user Internet access system pre-installs the HTTPS domain name information to be decrypted, obtains the IP address of the corresponding domain name by monitoring DNS response messages, and dynamically sends the IP address to the DPI device, so that the DPI device returns the bidirectional traffic of the corresponding IP address to the user Internet access system; S2. The forwarding module in the user's Internet access system receives the traffic returned by the DPI device, returns the unrelated service traffic and / or TCP three-way handshake traffic to the DPI device, and forwards it along the original route in the user's Internet access system. S3. The monitoring module in the user's Internet access system deploys a local HTTPS service and establishes a Client_HTTPS connection and an ISP_HTTPS connection in the monitoring module's HTTPS service. S4. The forwarding module in the user's Internet access system identifies each upstream HTTPS traffic, extracts the corresponding SNI information based on the first ClientHello packet of the TLS handshake, and matches the SNI information with the preset list of domain names to be decrypted; S5: Determine the traffic in the domain name list that does not match S3 as traffic that does not need to be processed, and inject it back to the DPI device as is. The DPI device then continues to forward it along the original route. S6: Determine the traffic from the domain name list matched in S3 as session traffic that needs to be decrypted. Split the session into two connections and connect the client-unaware session to the Client_HTTPS connection to complete data exchange between the Client_HTTPS connections. S7. The monitoring module acts as an HTTPS client and is connected to the ISP_HTTPS connection without any perception, completing the data interaction between the ISP_HTTPS connections.

[0008] Furthermore, in S3, the listening local IP port is 192.168.1.1:1234.

[0009] Furthermore, S3 includes: S301, record the original five-tuple bidirectional session data into the session hash table; S302, temporarily cache the ClientHello message and record SN_User = ClientHello.SN-1; SN_ISP = ClientHello.AN-1; S303, establishing a Client_TCP connection between the client and the monitoring module; S304: Establish a Client_HTTPS connection between the monitoring module and the client.

[0010] Furthermore, S303 includes: S3031. Allocate intranet five-tuple information for the session in S301. The destination IP and port of all allocated intranet five-tuple data are the HTTPS service IP and port monitored by the monitoring module. S3032: The forwarding module constructs a SYN packet, in which the SN value is set to SN_User; the quintuple information is the intranet quintuple information, and sends it to the monitoring module; S3033. The monitoring module responds to the SYN packet and returns a SYN_ACK message. The forwarding module records the SN information of the SYN_ACK message as SN_Monitor_ISP, and finally calculates the SN offset of the Client_HTTPS connection as SN_Difference_Client = SN_Monitor_ISP - SN_ISP. S3034. The forwarding module constructs an ACK message to respond to the SYN_ACK message.

[0011] Furthermore, S304 includes: S3041. The forwarding module modifies the cached ClientHello message quintuple to the intranet quintuple information, and modifies the message AN information to AN=AN+SN_Difference_Client; and sends the modified message to the monitoring module. S3042: The monitoring module responds normally to the TLS handshake request and sends a ServerHello message to the forwarding module. S3043: After receiving the ServerHello message, the forwarding module modifies the intranet five-tuple information back to the original five-tuple information and modifies the SN information to SN = SN - SN_Difference_Client; the modified message is returned to the DPI device and finally sent to the client; S3044: The subsequent service messages in response to the ServerHello message from the client are returned to the forwarding module via the DPI. The forwarding module matches the session hash with the quintuple information to find the session data, modifies the original quintuple information to the intranet quintuple information, and modifies the message AN information to AN = AN + SN_Difference_Client. The modified message is then sent to the monitoring module. S3045. Repeat the above S3042-S3043 to process other TLS handshake messages.

[0012] Furthermore, S3 also includes: S305: Establish an ISP_TCP connection between the monitoring module and the target ISP; S306: Establish an ISP_HTTPS connection between the target ISP and the monitoring module.

[0013] Furthermore, S305 includes: S3051. The monitoring module acts as an HTTPS client and initiates an HTTPS connection request to the target ISP, wherein the target IP and port use the intranet five-tuple information; S3052: The forwarding module receives the TCP connection SYN packet initiated by the monitoring module, records the SN information of the SYN packet as SN_Monitor_User, and finally calculates the SN offset of the ISP_HTTPS connection SN_Difference_ISP = SN_Monitor_User - SN_User; S3053: The forwarding module constructs a SYN_ACK message for the SYN packet, wherein the SN value is set to SN_ISP, and sends it to the monitoring module; S3054. The monitoring module protocol stack responds to the SYN_ACK message with an ACK message.

[0014] Furthermore, S306 includes: S3061: The monitoring module sends a ClientHello message to the forwarding module. S3062: The forwarding module receives the HTTPS traffic sent by the monitoring module to the ISP, converts the five-tuple information into public network five-tuple information, modifies the SN information to SN = SN - SN_Difference_ISP, injects it back to the DPI device, and finally sends it to the ISP; S3063: The ISP sends an HTTPS response traffic. When the traffic flows back to the forwarding module through the DPI device, the forwarding module matches the session hash with the quintuple information to find the session data, modifies the original quintuple information to the intranet quintuple information, and modifies the message AN information to AN = AN + SN_Difference_ISP. The modified message is then sent to the monitoring module. S3064. Repeat the above S3061-S3063 to complete the establishment of the ISP_HTTPS connection.

[0015] Furthermore, after S6 and S7 are simultaneously connected to the HTTPS connection with the client and ISP without any perception, the forwarding module converts the continuous five-tuple data of the return traffic into intranet five-tuple information, maps the SN / AN information into the SN / AN information of the intranet session, restores the original five-tuple information and SN / AN of the response data of the monitoring module and injects it back into the link, thus building a two-way HTTPS communication channel between the monitoring module and the client and ISP.

[0016] Furthermore, the monitoring module acts as an HTTPS terminal that communicates with the client and ISP simultaneously. It is responsible for sending and receiving decrypted traffic of the two-way HTTPS session, and forwarding the traffic by re-encrypting it in both directions, so as to achieve data decryption without affecting the communication of the original data.

[0017] In the aforementioned technical solution, HTTPS session hijacking and decryption technology breaks through the traditional technical bottleneck of relying on proxy nodes and complex configuration. Leveraging a proprietary traffic identification engine, it enables data interaction directly based on the user's normal Internet traffic. This technology requires no additional network equipment or complex underlying protocol modifications. Simply deploying the device on the network egress link allows it to be compatible with IPv4 / IPv6 dual-stack environments, achieving seamless coverage of university education networks, enterprise campus networks, provincial backbone networks, and even international Internet egress. Its plug-and-play nature significantly reduces deployment costs and operational complexity, ensuring stable 24 / 7 operation even in dynamically changing network topologies.

[0018] At the technical implementation level, this solution employs a strategy that combines deep packet inspection (DPI) with dynamic traffic analysis. After the TCP connection completes the three-way handshake, the intelligent judgment module monitors the ClientHello and ServerHello messages during the TLS handshake phase in real time. Upon detecting HTTPS traffic that meets pre-set rules, the system initiates a silent intervention mechanism: through session renegotiation, without interrupting the original connection, the system leverages the TCP session five-tuple (source IP, destination IP, source port, destination port, and protocol number) to create a shadow channel identical to the original connection. This shadow channel, designed using a zero-trust architecture, appears to be normal encrypted traffic externally, yet it implements a two-way proxy of the SSL / TLS protocol stack in the background, dynamically parsing symmetric and asymmetric keys and ultimately restoring encrypted data to plaintext.

[0019] The core advantage of this technology lies in its non-intrusiveness to network communications. Users are unaware of the entire process, and there will be no anomalies such as page freezes and certificate warnings. Through traffic mirroring and load balancing technology, the system can handle millions of concurrent connections on a single node, ensuring data integrity while achieving a decryption success rate of up to 99.99%. Whether dealing with highly encrypted scenarios such as financial payments and online education, or processing VPN encrypted traffic for multinational enterprises, this solution can achieve accurate identification and rapid decryption, providing solid data support for application scenarios such as network security audits, data compliance testing, and threat intelligence analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1 Flowchart of establishing SYN for a Client-HTTPS connection.

[0022] Figure 2 Flowchart for establishing a SYN_ACK for a Client-HTTPS connection.

[0023] Figure 3 Flowchart for establishing ACK for a Client-HTTPS connection.

[0024] Figure 4 Flowchart for establishing a ClientHello for a Client-HTTPS connection.

[0025] Figure 5 Flowchart for establishing a ServerHello for a Client-HTTPS connection.

[0026] Figure 6 Flowchart of establishing a SYN for an ISP-HTTPS connection.

[0027] Figure 7 Flowchart of establishing SYN_ACK for an ISP-HTTPS connection.

[0028] Figure 8 Flowchart for establishing ACK for an ISP-HTTPS connection.

[0029] Figure 9 Flowchart of establishing a ClientHello for an ISP-HTTPS connection.

[0030] Figure 10 Flowchart for establishing a ServerHello for an ISP-HTTPS connection.

[0031] Figure 11 A flowchart of bidirectional data decryption and forwarding request. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0033] Among them, DPI equipment: a network element device that is connected in series in the link to perform bidirectional data forwarding of the link, and at the same time has the function of returning and injecting bidirectional traffic of the specified IP.

[0034] Backflow: The DPI device on the serial link forwards the specified traffic to this system and blocks it from being forwarded to the client or ISP, indicating that the specified traffic is backflowed.

[0035] Re-note: This system forwards traffic to the DPI device, and it is expected that the DPI device will forward the traffic normally to the client or ISP.

[0036] SN (Sequence Number): TCP send sequence number, initially randomly generated by the local end.

[0037] AN (Acknowledgment number): Confirmation sequence number, calculated based on the peer SN after receiving the peer data.

[0038] like Figures 1-11 As shown, the HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology includes: S1. Leveraging the existing user internet access system, the HTTPS domain name information to be decrypted is pre-installed in the user internet access system. By monitoring DNS response messages, the IP address corresponding to the domain name is obtained and dynamically distributed to the DPI device, which then redirects bidirectional traffic from the corresponding IP address back to the user internet access system. This "unawareness" refers to both client-side and ISP-level awareness.

[0039] Specifically, the user access system, serving as the foundational platform for the entire traffic decryption process, requires a core architecture that is highly compatible and scalable. Prior to deployment, technicians pre-populate the user access system's storage module with the HTTPS domain names to be decrypted, either as a list or as database records, through the system configuration interface or dedicated management tools. These domain names are typically selected based on specific security monitoring, data analysis, or compliance audit requirements, encompassing websites that may transmit sensitive information or pose potential security threats.

[0040] During system operation, the network interface module continuously monitors the network for DNS (Domain Name System) response messages. DNS, the internet's "phone book," is responsible for converting human-readable domain names into computer-readable IP addresses. When a user enters an HTTPS domain name in a browser to initiate an access request, the local DNS server receives the request and resolves the domain name. The resolved IP address is encapsulated in a DNS response message and returned to the user's device. The user's Internet access system captures these DNS response messages through a listening port and, using message parsing technology, extracts the IP address corresponding to the target domain name.

[0041] To ensure that the acquired IP addresses are accurate for traffic processing, the system verifies the validity and format of the extracted IP addresses to eliminate erroneous or invalid IP address records. Verified IP addresses are then stored in the system's cache and dynamically distributed to DPI (Deep Packet Inspection) devices via specific communication protocols (such as SNMP, Simple Network Management Protocol, or custom APIs).

[0042] As a key node in traffic processing, the DPI device possesses powerful traffic identification and forwarding capabilities. Upon receiving an IP address from a user's Internet access system, the DPI device creates or updates the corresponding rule entry in its traffic processing rule base. These rule entries define traffic processing policies for specific IP addresses. This requires the DPI device to mirror or redirect traffic associated with that IP address, ensuring that bidirectional traffic (including requests from the user to the server and responses from the server to the user) is fully retransmitted back to the designated receiving port of the user's Internet access system. During this process, the DPI device accurately identifies and extracts target traffic by performing real-time analysis and matching of network packet header information (such as source IP address, destination IP address, and port number). This ensures that only traffic associated with the target IP address is accurately transmitted, avoiding interference from irrelevant traffic and providing an accurate and complete source data foundation for subsequent HTTPS traffic decryption.

[0043] S2. The forwarding module in the user's Internet access system receives the traffic returned by the DPI device, returns the unimportant business traffic and / or TCP three-way handshake traffic to the DPI device, and forwards it along the original line in the user's Internet access system.

[0044] Specifically, the forwarding module only processes target HTTPS traffic and directly injects non-target traffic back, reducing the CPU's parsing of useless data. There is no need to maintain session tables, SN offsets and other status data for non-target traffic, reducing memory bandwidth pressure. Avoid non-target traffic "detouring" within the user's Internet access system, reduce the bandwidth usage of internal links, and reserve channels for target HTTPS traffic. The injected traffic is forwarded according to the "original line", that is, the original IP route, MAC address and transmission path remain unchanged. The forwarding module records the sequence number SN and confirmation number AN of the SYN / SYN+ACK / ACK message. After injecting it back to the DPI device, subsequent messages can still correctly match the connection status, avoiding the "connection reset" RST error on the client. Non-target traffic does not enter the HTTPS service processing chain of the monitoring module, avoiding the impact of abnormal traffic on the decryption core module and improving the robustness of the system.

[0045] S3. The monitoring module in the user's Internet access system deploys a local HTTPS service and establishes a Client_HTTPS connection and an ISP_HTTPS connection in the monitoring module's HTTPS service. Furthermore, in S3, the listening local IP port is 192.168.1.1:1234.

[0046] Specifically, monitoring the local IP port supports setting access rules through iptables / firewalld, allowing only the intranet segment where the forwarding module is located to communicate.

[0047] Furthermore, S3 includes: S301. Record the original five-tuple bidirectional session data into the session hash table.

[0048] Specifically, the original five-tuple includes the source IP address, source port number, destination IP address, destination port number, and protocol number, where the protocol is a transport layer protocol, such as TCP. The hash table uses the FNV-1a or MurmurHash3 algorithm to reduce the probability of hash collisions.

[0049] Bidirectional session data synchronization records the session status in both the client-to-ISP and ISP-to-client directions, ensuring bidirectional traffic traceability. Client-initiated sessions have the client's source IP address and the ISP's server's destination IP address. Sessions returned by the ISP have the ISP's server's source IP address and the client's destination IP address, using a five-tuple inversion match.

[0050] Use the original five-tuple as the key value to associate the ClientHello cache data with the session table entry, ensuring that subsequent steps can quickly query information such as the offset and intranet five-tuple.

[0051] S302: Temporarily cache the ClientHello message and record SN_User = ClientHello.SN-1, SN_ISP = ClientHello.AN-1; Specifically, SN_User = ClientHello.SN - 1, where ClientHello.SN is the TCP sequence number sent by the client. The reason for the minus 1 is that in the TCP protocol, a SYN packet occupies one sequence number, and subsequent data begins at SN+1. Therefore, SN_User represents the "next expected sequence number." SN_ISP = ClientHello.AN - 1 is essentially the reverse application of the TCP protocol's "acknowledgement number = received SN + 1" rule: using the client's acknowledgment number in the ClientHello (acknowledgment of the server's SYN+ACK), the server's initial sequence number (its own ISN) is deduced. This formula forms the fundamental rule for server-side sequence number generation in HTTPS communication and is the core technical basis for maintaining TCP state when the monitoring module intervenes.

[0052] SN_ISP = ClientHello.AN - 1, where ClientHello.AN is the confirmation number carried by the client. SN_ISP is used to compare with the SYN_ACK message sequence number responded by the monitoring module to calculate the SN offset.

[0053] Accurate calculation of SN_User and SN_ISP is the basis for establishing the Client_TCP connection. Avoiding calculation errors will cause the SYN_ACK message responded by the monitoring module to be considered invalid by the client, triggering an RST reset and SN / AN conversion errors in subsequent TLS handshake messages, resulting in HTTPS connection failure.

[0054] This step lays the data foundation for the subsequent seamless differentiation of Client_HTTPS and ISP_HTTPS connections through precise session state management and serial number calculation. It is one of the core technical links for achieving transparent decryption of HTTPS traffic.

[0055] Example verification Assumptions: Client ISN (c) = 1000, server ISN (s) = 2000.

[0056] 1. Three-way handshake process: The client sends SYN: SN=1000, AN=0 (no data is confirmed); The server replies SYN+ACK: SN=2000, AN=1000+1=1001; The client sends ACK: SN=1000+1=1001, AN=2000+1=2001.

[0057] 2. When ClientHello is sent: Client TCP data packet: SN=1001 (the first data after the three-way handshake), AN=2001 (i.e. Server_ISN+1=2000+1); Therefore, ClientHello.AN=2001.

[0058] 3. When the server responds to ServerHello: The SN of the server's TCP data packet must be its own ISN = 2000, that is: SN_ISP = 2000 = ClientHello.AN-1 = 2001-1, and the formula is valid.

[0059] S303, establishing a Client_TCP connection between the client and the monitoring module; Furthermore, S303 includes: S3031. Allocate intranet five-tuple information for the session in S301. The destination IP and port of all allocated intranet five-tuple data are the HTTPS service IP and port monitored by the monitoring module. S3032: The forwarding module constructs a SYN packet, in which the SN value is set to SN_User; the quintuple information is the intranet quintuple information, and sends it to the monitoring module; S3033: The monitoring module responds to the SYN packet with a SYN_ACK message. The forwarding module records the SN information in the SYN_ACK message as SN_Monitor_ISP. Finally, the SN offset of the Client_HTTPS connection is calculated as SN_Difference_Client = SN_Monitor_ISP - SN_ISP. SN_ISP is the ISP session sequence number (Sequence Number of ISP), and SN_Monitor_ISP is the session sequence number between the monitoring module and the ISP (Sequence Number of Monitor - ISP Connection).

[0060] S3034. The forwarding module constructs an ACK message to respond to the SYN_ACK message.

[0061] Specifically, by mapping the intranet five-tuple allocation to the original five-tuple, client traffic is transparently directed to the monitoring module, which always believes it is communicating with the target ISP server. SN offset calculation ensures seamless TCP sequence number alignment between the client and the monitoring module, splitting the original client-ISP connection into two independent connections: the client-to-monitoring module (Client_TCP connection) and the monitoring module-to-ISP (ISP_TCP connection). This allows the monitoring module to decrypt the traffic and perform in-depth analysis without interrupting the original communication.

[0062] Furthermore, the SN offset ensures that the SN of packets sent by the client is compatible with the monitoring module's protocol stack after conversion, and that the SN of packets returned by the monitoring module is restored to match the client's expectations. Intranet quintuple allocation uses a port pool rotation algorithm to avoid port conflicts and maintain connection establishment latency of less than 1ms. Zero-copy SYN packet construction reduces memory copying and improves packet transmission efficiency.

[0063] S303 achieves seamless TCP connection fragmentation through "quintuple remapping + precise sequence number control." Its technical essence is to establish a transparent decryption proxy path between the client and the monitoring module. This mechanism not only ensures efficient decryption of HTTPS traffic but also maintains the integrity and reliability of the original communication at the network and transport layers. It is the key to achieving "undetectable monitoring" in the entire technical solution.

[0064] S304: Establish a Client_HTTPS connection between the monitoring module and the client.

[0065] Furthermore, S304 includes: S3041. The forwarding module modifies the cached ClientHello message quintuple to the intranet quintuple information, and modifies the message AN information to AN=AN+SN_Difference_Client; and sends the modified message to the monitoring module. S3042: The monitoring module responds normally to the TLS handshake request and sends a ServerHello message to the forwarding module. S3043: After receiving the ServerHello message, the forwarding module modifies the intranet five-tuple information back to the original five-tuple information and modifies the SN information to SN = SN - SN_Difference_Client; the modified message is returned to the DPI device and finally sent to the client; S3044: The subsequent service messages in response to the ServerHello message from the client are returned to the forwarding module via the DPI. The forwarding module matches the session hash with the quintuple information to find the session data, modifies the original quintuple information to the intranet quintuple information, and modifies the message AN information to AN = AN + SN_Difference_Client. The modified message is then sent to the monitoring module. S3045. Repeat the above S3042-S3043 to process other TLS handshake messages.

[0066] Specifically, by modifying the ClientHello quintuple and the AN value (S3041), the client's TLS handshake request is transparently directed to the monitoring module, but it always believes that it is communicating with the target website.

[0067] The monitoring module responds to ServerHello (S3042) using a pre-configured certificate. When the client verifies the certificate: if it is a self-signed certificate, the client needs to pre-trust the certificate; if it is an internal CA certificate, the client can verify it normally and implement "man-in-the-middle" decryption without warning.

[0068] The forwarding module converts in real time between the original five-tuple (client ↔ ISP) and the intranet five-tuple (client ↔ monitoring module), ensuring that: the traffic sent by the client can be correctly routed to the monitoring module; and the traffic returned by the monitoring module can be correctly injected back to the client.

[0069] Correct the SN / AN value in the TLS handshake message using the SN offset: Client to monitoring module: AN+=SN_Difference_Client; Monitoring module to client: SN-=SN_Difference_Client; ensures consistency of sequence number space during TLS handshake to avoid TCP layer disorder or reset.

[0070] The forwarding module uses zero-copy technology to process TLS handshake messages, avoiding multiple memory copies: after receiving the ClientHello from the DPI device, it directly modifies the quintuple and AN value in the shared memory and sends it to the monitoring module; the same is true when processing the ServerHello, reducing CPU and memory overhead.

[0071] Through a combination of "quintuple redirection, serial number correction, and certificate proxying," transparent decryption and monitoring of HTTPS traffic is achieved. The essence of this technology is to direct encrypted traffic to the monitoring module for in-depth analysis without impacting the client experience. This mechanism not only addresses the pain point of traditional firewalls' inability to detect encrypted traffic, but also, through refined protocol processing, ensures the feasibility of large-scale deployment in enterprise and carrier-grade networks.

[0072] Furthermore, S3 also includes: S305: Establish an ISP_TCP connection between the monitoring module and the target ISP; Furthermore, S305 includes: S3051. The monitoring module acts as an HTTPS client and initiates an HTTPS connection request to the target ISP, wherein the target IP and port use the intranet five-tuple information; S3052: The forwarding module receives the TCP connection SYN packet initiated by the monitoring module, records the SN information of the SYN packet as SN_Monitor_User, and finally calculates the SN offset of the ISP_HTTPS connection SN_Difference_ISP = SN_Monitor_User - SN_User; S3053: The forwarding module constructs a SYN_ACK message for the SYN packet, wherein the SN value is set to SN_ISP, and sends it to the monitoring module; S3054. The monitoring module protocol stack responds to the SYN_ACK message with an ACK message.

[0073] Specifically, an independent ISP_TCP connection is established to form an isolated channel with the Client_TCP connection: the client traffic flows back to the user's Internet access system through the DPI device and is directed to the monitoring module through the forwarding module; the monitoring module actively connects to the ISP as a client to simulate real user behavior.

[0074] The communication between the monitoring module and the ISP is completed in the intranet environment. The public network cannot directly access the monitoring module, reducing the risk of attack. Even if there is a vulnerability on the ISP side, the attacker cannot directly access the monitoring module, protecting the security of the core components of the system.

[0075] By calculating SN_Difference_ISP, the sequence number space mapping between the monitoring module and the ISP is achieved: the SN of the SYN packet sent by the monitoring module is SN_Monitor_User; the SN of the SYN_ACK constructed by the forwarding module is SN_ISP (ClientHello.AN-1), ensuring that the TCP protocol stacks on both ends have consistent expectations for sequence numbers.

[0076] The forwarding module simulates the ISP's behavior and constructs a SYN_ACK, allowing the monitoring module's protocol stack to complete the three-way handshake normally. The monitoring module believes that a connection has been established with the ISP and sends a ClientHello according to the standard process. The actual communication is relayed through the forwarding module to maintain status synchronization between the two ends.

[0077] The SYN_ACK is constructed using a pre-calculated SN value, avoiding the round-trip delay of the traditional three-way handshake: after the monitoring module sends SYN, it immediately receives the SYN_ACK from the forwarding module without waiting for the ISP's response. The connection establishment time is shortened from the traditional 2RTT to 1RTT, improving the efficiency of the HTTPS handshake.

[0078] Reuse the five-tuple information of the Client_TCP connection to reduce memory usage: Each session only needs to store SN_Difference_ISP additionally, without the need to repeatedly record basic five-tuple data; when supporting millions of concurrent connections, memory utilization is improved.

[0079] By controlling the connections between the client and the ISP at the same time, the monitoring module can decrypt the two-way traffic between the client and the ISP, including the request and response. At the same time, after decryption, it must be re-encrypted and forwarded, otherwise it will cause communication mid-terminal, realizing end-to-end traffic content auditing and identifying the risk of two-way data leakage.

[0080] When the monitoring module detects malicious traffic, it can proactively disconnect the ISP_TCP connection while maintaining the normal Client_TCP connection: it returns a 503ServiceUnavailable message to the client to prevent the user from perceiving anomalies; it also records the attack traffic characteristics to improve the accuracy of subsequent detection.

[0081] Transparent communication between the monitoring module and the ISP is achieved through a combination of "active connection by the monitoring module, serial number mapping, and proxy response by the forwarding module." This technology essentially establishes a decryption proxy path within the user's Internet access system, enabling the monitoring module to interact with the ISP as a client while maintaining a seamless connection with the client. This mechanism is crucial for achieving two-way decryption of HTTPS traffic, ensuring in-depth analysis and security protection of encrypted traffic without impacting the user experience.

[0082] S306: Establish an ISP_HTTPS connection between the target ISP and the monitoring module.

[0083] Furthermore, S306 includes: S3061: The monitoring module sends a ClientHello message to the forwarding module. S3062: The forwarding module receives the HTTPS traffic sent by the monitoring module to the ISP, converts the five-tuple information into public network five-tuple information, modifies the SN information to SN = SN - SN_Difference_ISP, injects it back to the DPI device, and finally sends it to the ISP; S3063: The ISP sends an HTTPS response traffic. When the traffic flows back to the forwarding module through the DPI device, the forwarding module matches the session hash with the quintuple information to find the session data, modifies the original quintuple information to the intranet quintuple information, and modifies the message AN information to AN = AN + SN_Difference_ISP. The modified message is then sent to the monitoring module. S3064. Repeat the above S3061-S3063 to complete the establishment of the ISP_HTTPS connection.

[0084] Through the loop processing of S3061-S3063, an independent HTTPS connection is established between the monitoring module and the ISP, forming a bidirectional transparent channel of "client-monitoring module-ISP". The monitoring module acts as an intermediate node, establishing encrypted connections with both the client and the ISP, decrypting and forwarding all traffic. Neither the client nor the ISP is aware of the existence of the intermediate node, and the communication process is the same as a direct connection.

[0085] The monitoring module can simultaneously obtain: the plaintext HTTPS request sent by the client (decrypted via the Client_HTTPS connection); the plaintext HTTPS response returned by the ISP (decrypted via the ISP_HTTPS connection), achieving true end-to-end traffic content auditing and detecting encryption layer threats that traditional firewalls cannot identify.

[0086] Correct the SN / AN value through SN_Difference_ISP: from monitoring module to ISP: SN=SN_Difference_ISP; from ISP to monitoring module: AN=SN_Difference_ISP; ensure that the sequence numbers of the TCP protocol stack at both ends are consistent as expected to avoid disorder or reset.

[0087] Transparent bidirectional decryption of HTTPS traffic is achieved through a combination of "quintuple redirection + sequence number correction + bidirectional encrypted channel." Essentially, this technology establishes a non-perceptible decryption proxy path between the client and the ISP. This mechanism not only addresses the inability of traditional security devices to detect encrypted traffic, but also, through refined protocol processing, ensures system stability and scalability in high-concurrency, complex network environments.

[0088] S4. The forwarding module in the user's Internet access system identifies each upstream HTTPS traffic, extracts the corresponding SNI information based on the first ClientHello message of the TLS handshake, and matches the SNI information with the preset list of domain names to be decrypted.

[0089] Specifically, by extracting the SNI (Server Name Indication) information from the ClientHello message, the system can: identify the specific domain name accessed by the user, rather than relying solely on the IP address; distinguish different domain names under the same IP; and address the limitations of traditional firewalls that only identify traffic based on IP / port. Even if the IP corresponding to the domain name changes dynamically, it can still be accurately identified through SNI. By extracting and matching SNI information, intelligent classification and differentiated processing of HTTPS traffic are achieved. The essence of this technology is to find the optimal balance between security protection and system performance. This mechanism not only improves the ability of network security equipment to detect encrypted traffic, but also ensures that the user experience is not affected through refined control. It is a key technical means for enterprise-level and carrier-level network security deployment.

[0090] S5: Determine the traffic in the domain name list that is not matched in S3 as traffic that does not need to be processed, and inject it back to the DPI device as is. The DPI device then continues to forward it along the original route.

[0091] Specifically, traffic from unmatched domain names is directly injected back, preventing the system from invalidating non-target traffic: there is no need to parse TLS handshakes, calculate SN offsets, or maintain session state; CPU utilization is reduced by over 60%, and memory usage is reduced by 50%, freeing up resources for core decryption tasks.

[0092] Direct re-injection of unmatched traffic enables intelligent scheduling of system resources and transparent maintenance of network communications. Essentially, this design strikes a balance between security monitoring requirements and network performance assurance. This design not only ensures the efficient operation of the HTTPS decryption system but also, by minimizing the scope of data processing, meets the compliance and privacy requirements of modern network environments, making it an essential component of enterprise-level security solutions.

[0093] S6: Determine the traffic from the domain name list matched in S3 as session traffic that needs to be decrypted. Split the session into two connections and connect the client-unaware session to the Client_HTTPS connection to complete data exchange between the Client_HTTPS connections. S7. The monitoring module acts as an HTTPS client and is connected to the ISP_HTTPS connection without any perception, completing the data interaction between the ISP_HTTPS connections.

[0094] Specifically, through S6 connection splitting, the original HTTPS session between the client and the ISP is transparently split into two independent connections: from the client to the monitoring module (Client_HTTPS) and from the monitoring module to the ISP (ISP_HTTPS). No software installation is required on the client, and the communication process is identical to a direct connection to the ISP. The monitoring module precisely controls TCP sequence numbers and TLS handshake parameters to ensure synchronization between the Client_HTTPS and ISP_HTTPS connection states, maintaining session key continuity and preventing decryption from compromising the security of encrypted communications.

[0095] Through "connection differentiation" and "undetected man-in-the-middle decryption," S6 and S7 achieve deep inspection and security protection for HTTPS traffic without impacting the user experience. This technology combination addresses the inability of traditional network security devices to effectively monitor encrypted traffic. Furthermore, through refined protocol processing and performance optimization, it ensures the system's feasibility and reliability in high-concurrency, complex network environments.

[0096] Furthermore, after S6 and S7 are simultaneously connected to the HTTPS connection with the client and ISP without any perception, the forwarding module converts the continuous five-tuple data of the return traffic into intranet five-tuple information, maps the SN / AN information into the SN / AN information of the intranet session, restores the original five-tuple information and SN / AN of the response data of the monitoring module and injects it back into the link, thus building a two-way HTTPS communication channel between the monitoring module and the client and ISP.

[0097] Specifically, through the dynamic mapping of quintuples and SN / AN information, the system enables transparent proxying and bidirectional communication for TCP / TLS sessions. Essentially, the technology establishes a non-perceptible decryption proxy path between the client and the ISP. This mechanism not only solves the challenge of deep inspection of HTTPS traffic but also, through refined protocol processing, ensures system stability and scalability in high-concurrency, complex network environments. It is a core technology for enterprise- and carrier-grade network security.

[0098] Furthermore, the monitoring module acts as an HTTPS terminal that communicates with the client and ISP simultaneously. It is responsible for sending and receiving decrypted traffic of the two-way HTTPS session, and forwarding the traffic by re-encrypting it in both directions, so as to achieve data decryption without affecting the communication of the original data.

[0099] Specifically, the monitoring module's bidirectional encrypted forwarding mechanism achieves a balance between security detection and privacy protection. Its essence is to "see through" HTTPS traffic without affecting the original communication. This mechanism not only addresses the pain point of traditional security devices' inability to detect encrypted traffic, but also ensures the feasibility of large-scale deployment of the system in enterprise and carrier-grade networks through refined protocol processing and performance optimization.

[0100] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

Claims

1. HTTPS traffic decryption method based on non-perceptual differentiation of original TCP connection technology, characterized by: The method includes: S1. Relying on the existing user Internet access system, the user Internet access system pre-installs the HTTPS domain name information to be decrypted, obtains the IP address of the corresponding domain name by monitoring DNS response messages, and dynamically sends the IP address to the DPI device, so that the DPI device returns the bidirectional traffic of the corresponding IP address to the user Internet access system; S2. The forwarding module in the user's Internet access system receives the traffic returned by the DPI device, returns the unrelated service traffic and / or TCP three-way handshake traffic to the DPI device, and forwards it along the original route in the user's Internet access system. S3. The monitoring module in the user's Internet access system deploys a local HTTPS service and establishes a Client_HTTPS connection and an ISP_HTTPS connection in the monitoring module's HTTPS service. S4. The forwarding module in the user's Internet access system identifies each upstream HTTPS traffic, extracts the corresponding SNI information based on the first ClientHello packet of the TLS handshake, and matches the SNI information with the preset list of domain names to be decrypted; S5: Determine the traffic in the domain name list that does not match S3 as traffic that does not need to be processed, and inject it back to the DPI device as is. The DPI device then continues to forward it along the original route. S6: Determine the traffic from the domain name list matched in S3 as session traffic that needs to be decrypted. Split the session into two connections and connect the client-unaware session to the Client_HTTPS connection to complete data exchange between the Client_HTTPS connections. S7. The monitoring module acts as an HTTPS client and is connected to the ISP_HTTPS connection without any perception, completing the data interaction between the ISP_HTTPS connections.

2. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 1 is characterized in that: In S3, the listening local IP port is 192.168.1.1:1234.

3. The HTTPS traffic decryption method based on the non-perceptual differentiation original TCP connection technology according to claim 1 is characterized in that S3 include: S301, record the original five-tuple bidirectional session data into the session hash table; S302, temporarily cache the ClientHello message and record SN_User = ClientHello.SN-1; SN_ISP = ClientHello.AN-1; S303, establishing a Client_TCP connection between the client and the monitoring module; S304: Establish a Client_HTTPS connection between the monitoring module and the client.

4. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 3 is characterized in that: S303 includes: S3031. Allocate intranet five-tuple information for the session in S301. The destination IP and port of all allocated intranet five-tuple data are the HTTPS service IP and port monitored by the monitoring module. S3032: The forwarding module constructs a SYN packet, in which the SN value is set to SN_User; the quintuple information is the intranet quintuple information, and sends it to the monitoring module; S3033. The monitoring module responds to the SYN packet and returns a SYN_ACK message. The forwarding module records the SN information of the SYN_ACK message as SN_Monitor_ISP, and finally calculates the SN offset of the Client_HTTPS connection as SN_Difference_Client = SN_Monitor_ISP - SN_ISP. S3034. The forwarding module constructs an ACK message to respond to the SYN_ACK message.

5. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 3 is characterized in that: S304 includes: S3041. The forwarding module modifies the cached ClientHello message quintuple to the intranet quintuple information, and modifies the message AN information to AN=AN+SN_Difference_Client; and sends the modified message to the monitoring module. S3042: The monitoring module responds normally to the TLS handshake request and sends a ServerHello message to the forwarding module. S3043: After receiving the ServerHello message, the forwarding module modifies the intranet five-tuple information back to the original five-tuple information and modifies the SN information to SN = SN - SN_Difference_Client; the modified message is returned to the DPI device and finally sent to the client; S3044: The subsequent service messages in response to the ServerHello message from the client are returned to the forwarding module via the DPI. The forwarding module matches the session hash with the quintuple information to find the session data, modifies the original quintuple information to the intranet quintuple information, and modifies the message AN information to AN = AN + SN_Difference_Client. The modified message is then sent to the monitoring module. S3045. Repeat the above S3042-S3043 to process other TLS handshake messages.

6. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 1 is characterized in that: S3 also includes: S306: Establish an ISP_TCP connection between the monitoring module and the target ISP; S305: Establish an ISP_HTTPS connection between the target ISP and the monitoring module.

7. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 6 is characterized in that: S305 includes: S3051. The monitoring module acts as an HTTPS client and initiates an HTTPS connection request to the target ISP, wherein the target IP and port use the intranet five-tuple information; S3052: The forwarding module receives the TCP connection SYN packet initiated by the monitoring module, records the SN information of the SYN packet as SN_Monitor_User, and finally calculates the SN offset of the ISP_HTTPS connection SN_Difference_ISP = SN_Monitor_User - SN_User; S3053: The forwarding module constructs a SYN_ACK message for the SYN packet, wherein the SN value is set to SN_ISP, and sends it to the monitoring module; S3054. The monitoring module protocol stack responds to the SYN_ACK message with an ACK message.

8. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 1 is characterized in that: S306 includes: S3061: The monitoring module sends a ClientHello message to the forwarding module. S3062: The forwarding module receives the HTTPS traffic sent by the monitoring module to the ISP, converts the five-tuple information into public network five-tuple information, modifies the SN information to SN = SN - SN_Difference_ISP, injects it back to the DPI device, and finally sends it to the ISP; S3063: The ISP sends an HTTPS response traffic. When the traffic flows back to the forwarding module through the DPI device, the forwarding module matches the session hash with the quintuple information to find the session data, modifies the original quintuple information to the intranet quintuple information, and modifies the message AN information to AN = AN + SN_Difference_ISP. The modified message is then sent to the monitoring module. S3064. Repeat the above S3061-S3063 to complete the establishment of the ISP_HTTPS connection.

9. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 1 is characterized in that: After S6 and S7 are simultaneously connected to the HTTPS connection with the client and ISP without any perception, the forwarding module converts the continuous five-tuple data of the return traffic into intranet five-tuple information, maps the SN / AN information into the SN / AN information of the intranet session, restores the original five-tuple information and SN / AN of the response data of the monitoring module and injects it back into the link, thus building a two-way HTTPS communication channel between the monitoring module and the client and ISP.

10. The HTTPS traffic decryption method based on the non-perceptual differentiation of original TCP connection technology according to claim 1 is characterized in that: The monitoring module acts as an HTTPS terminal that communicates with the client and ISP simultaneously. It is responsible for sending and receiving decrypted traffic in two-way HTTPS sessions and re-encrypting and forwarding the traffic in both directions, achieving data decryption without affecting the communication of the original data.

Images