Encryption and decryption flow control method based on transparent proxy

Abstract

The invention discloses an encryption and decryption flow control method based on transparent proxy, which belongs to the technical field of network security, and comprises the following steps of: a, judging whether internal network port flow and external network port flow are white lists ip or not through a transparent proxy module, and if so, directly skipping a decryption module to enable the internal network port flow to be directly connected; if not, transmitting to a decryption module; b, a decryption module establishes two connections to form a transfer station, one connection is connected to a client, the other connection is connected to a service server, and whether the flow is an SSL / TLS protocol is judged through the decryption module; and c, transmitting the plaintext traffic and data decrypted by the decryption module to a traffic analysis module, processing the data by the traffic analysis module, storing a processing result into a shared memory, restoring the processing result to correspond to the two connections after the decryption module obtains the processing result, and releasing or blocking the two connections. According to the method, anti-crawler can be overcome, normal communication is guaranteed, and network traffic can be effectively prevented from being blocked in the whole processing process.

Description

Technical Field

[0001] The invention relates to the technical field of network security, and in particular to an encryption and decryption flow control method based on a transparent proxy. Background Art

[0002] The role of a transparent proxy is to plug the company network's inbound and outbound cables into one port on a device, and the external network cable into another port on the device. If the device is not running a special program, the internal and external networks cannot communicate normally. Then install a transparent proxy program to forward the traffic between the internal and external network ports to each other, so that the networks can communicate and forward a copy of the traffic to the network security program, which can analyze the network and identify viruses and sensitive information.

[0003] Existing solutions cannot identify encrypted traffic. For example, if a website uses the SSL protocol and the page content is encrypted by SSL, if the page contains sensitive information, it will not be recognized. The SSL protocol, or Secure Sockets Layer protocol, is designed to achieve secure data transmission on the insecure Internet. It uses encryption or authentication technology to protect the transmitted privacy data. The SSL protocol itself is a security protocol with a high degree of openness.

[0004] A Chinese patent document with publication number CN117938447A and publication date April 26, 2024 discloses a blocking method based on detection of https encrypted traffic, characterized in that it is performed by an IP blocking device serially connected to the network main intersection and the core switch, including the following steps: S1, configuring the business service information of the IP blocking device; S2. Determine the client source IP and handshake request type according to the request and handshake packet sent by the client, establish a connection, transmit data, and obtain the plaintext content of the transmitted data; S3, perform threat identification and detection on the plain text content of the transmitted data; S4. Block the source IP or target IP of the data source detected to have threatening behavior.

[0005] The blocking method based on the detection of https encrypted traffic disclosed in the patent document can decrypt, detect threats and block unknown traffic of the transmitted HTTPS protocol through a transparent proxy at the transport layer and a reverse proxy at the application layer. However, since the browser fingerprint is not simulated when the transit connection with the business server is made, if the business browser implements an anti-crawler mechanism, the connection will be directly rejected and normal communication will not be possible. In addition, since asynchronous processing is not used, the entire processing process will block network traffic. Summary of the invention

[0006] In order to overcome the defects of the above-mentioned prior art, the present invention provides an encryption and decryption flow control method based on transparent proxy. The present invention can overcome anti-crawler and ensure normal communication. The entire processing process can effectively avoid blocking network flow.

[0007] The present invention is achieved through the following technical solutions: The encryption and decryption flow control method based on transparent proxy includes the following steps: a. Use the transparent proxy module to determine whether the intranet port traffic and the external network port traffic are whitelisted IPs. If so, the decryption module is skipped directly to connect the intranet port traffic directly; otherwise, the traffic is transmitted to the decryption module; b. The decryption module establishes two connections to form a transfer station, one to the client and the other to the business server. The decryption module determines whether the traffic is SSL / TLS protocol; c. The plaintext traffic and the data decrypted by the decryption module are transmitted to the traffic analysis module. After being processed by the traffic analysis module, the processing results are stored in the shared memory. After the decryption module obtains the processing results, it restores the processing results to correspond to the two connections and releases or blocks them.

[0008] In the step a, the transparent proxy module is used to connect the traffic of the internal network port and the traffic of the external network port.

[0009] In the step a, the decryption module is used to decrypt the SSL / TLS payload data.

[0010] In the step b, judging whether the traffic is the SSL / TLS protocol through the decryption module means that when the traffic is judged to be the SSL / TLS protocol, the traffic session is marked, and the two connections enable the SSL / TLS module for encrypted communication; when the traffic is judged not to be the SSL / TLS protocol, the two connections use TCP communication.

[0011] The SSL / TLS module is used to build the SSL / TLS protocol and encrypt and decrypt messages.

[0012] In the step b, the connection to the business server simulates the browser fingerprint when client_hello occurs.

[0013] The simulated browser fingerprint means that after the business server receives the client_hello sent by the SSL / TLS module, the business server calculates ja3 based on the fields in the client_hello message and compares it with the browser's inherent ja3. If it is determined that it is not from the browser, the connection is disconnected. The SSL / TLS module modifies the fields in the sent client_hello message so that the ja3 calculated by the business server is consistent with that of the browser, and the connection is restored.

[0014] In step c, the traffic analysis module is used to identify sensitive data in plaintext traffic.

[0015] In the step c, the traffic analysis module is used to store the processing results in the shared memory.

[0016] In step c, the shared memory is used for the decryption module process and the traffic analysis module process to access together.

[0017] The SSL / TLS described in the present invention refers to the transport layer security protocol.

[0018] The client_hello described in the present invention refers to the first message of the transport layer security protocol for handshake.

[0019] The ja3 described in the present invention refers to a character string that is hashed by extracting part of the information from client_hello.

[0020] The key described in the present invention refers to a unique identifier generated by a decryption module.

[0021] The beneficial effects of the present invention are mainly manifested in the following aspects: 1. Compared with the prior art, the present invention can overcome anti-crawler problems and ensure normal communication. The entire processing process can effectively avoid blocking network traffic.

[0022] 2. The present invention, through the SSL / TLS module, can decrypt the SSL / TLS encrypted traffic for analysis and simulate browser behavior, thereby ensuring the security of network communications.

[0023] 3. The present invention uses a traffic analysis module to identify sensitive data in plaintext traffic, and can block sensitive traffic in time to protect network security.

[0024] 4. The present invention is based on an efficient event notification mechanism and a state machine to isolate the traffic of the internal network port and the external network port so that they do not interact directly. The internal network port and the external network port communicate indirectly through a decryption module, which can flexibly control network traffic.

[0025] 5. The present invention adds a whitelist function, which can release specified IP traffic, reduce the amount of calculation and improve efficiency.

[0026] 6. The present invention adopts modular design, unifies the interaction interfaces of each module, decouples the code, enables quick replacement of modules, and improves maintenance convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Figure 1 It is a flowchart of the present invention; Figure 2 is a system structure diagram of the present invention; Figure 3 It is the work flow chart of the present invention. DETAILED DESCRIPTION

[0028] Example 1 The encryption and decryption flow control method based on transparent proxy includes the following steps: a. Use the transparent proxy module to determine whether the intranet port traffic and the external network port traffic are whitelisted IPs. If so, the decryption module is skipped directly to connect the intranet port traffic directly; otherwise, the traffic is transmitted to the decryption module; b. The decryption module establishes two connections to form a transfer station, one to the client and the other to the business server. The decryption module determines whether the traffic is SSL / TLS protocol; c. The plaintext traffic and the data decrypted by the decryption module are transmitted to the traffic analysis module. After being processed by the traffic analysis module, the processing results are stored in the shared memory. After the decryption module obtains the processing results, it restores the processing results to correspond to the two connections and releases or blocks them.

[0029] This embodiment is the most basic implementation method. Compared with the existing technology, it can overcome anti-crawler problems and ensure normal communication. The entire processing process can effectively avoid blocking network traffic.

[0030] Example 2 The encryption and decryption flow control method based on transparent proxy includes the following steps: a. Use the transparent proxy module to determine whether the intranet port traffic and the external network port traffic are whitelisted IPs. If so, the decryption module is skipped directly to connect the intranet port traffic directly; otherwise, the traffic is transmitted to the decryption module; b. The decryption module establishes two connections to form a transfer station, one to the client and the other to the business server. The decryption module determines whether the traffic is SSL / TLS protocol; c. The plaintext traffic and the data decrypted by the decryption module are transmitted to the traffic analysis module. After being processed by the traffic analysis module, the processing results are stored in the shared memory. After the decryption module obtains the processing results, it restores the processing results to correspond to the two connections and releases or blocks them.

[0031] In the step a, the transparent proxy module is used to connect the traffic of the internal network port and the traffic of the external network port.

[0032] In the step a, the decryption module is used to decrypt the SSL / TLS payload data.

[0033] In the step b, judging whether the traffic is the SSL / TLS protocol through the decryption module means that when the traffic is judged to be the SSL / TLS protocol, the traffic session is marked, and the two connections enable the SSL / TLS module for encrypted communication; when the traffic is judged not to be the SSL / TLS protocol, the two connections use TCP communication.

[0034] The SSL / TLS module is used to build the SSL / TLS protocol and encrypt and decrypt messages.

[0035] This embodiment is a preferred implementation method. Through the SSL / TLS module, the SSL / TLS encrypted traffic can be decrypted for analysis and browser behavior can be simulated, thereby ensuring the security of network communications.

[0036] Example 3 The encryption and decryption flow control method based on transparent proxy includes the following steps: a. Use the transparent proxy module to determine whether the intranet port traffic and the external network port traffic are whitelisted IPs. If so, the decryption module is skipped directly to connect the intranet port traffic directly; otherwise, the traffic is transmitted to the decryption module; b. The decryption module establishes two connections to form a transfer station, one to the client and the other to the business server. The decryption module determines whether the traffic is SSL / TLS protocol; c. The plaintext traffic and the data decrypted by the decryption module are transmitted to the traffic analysis module. After being processed by the traffic analysis module, the processing results are stored in the shared memory. After the decryption module obtains the processing results, it restores the processing results to correspond to the two connections and releases or blocks them.

[0037] In the step a, the transparent proxy module is used to connect the traffic of the internal network port and the traffic of the external network port.

[0038] In the step a, the decryption module is used to decrypt the SSL / TLS payload data.

[0039] In the step b, judging whether the traffic is the SSL / TLS protocol through the decryption module means that when the traffic is judged to be the SSL / TLS protocol, the traffic session is marked, and the two connections enable the SSL / TLS module for encrypted communication; when the traffic is judged not to be the SSL / TLS protocol, the two connections use TCP communication.

[0040] The SSL / TLS module is used to build the SSL / TLS protocol and encrypt and decrypt messages.

[0041] In the step b, the connection to the business server simulates the browser fingerprint when client_hello occurs.

[0042] The simulated browser fingerprint means that after the business server receives the client_hello sent by the SSL / TLS module, the business server calculates ja3 based on the fields in the client_hello message and compares it with the browser's inherent ja3. If it is determined that it is not from the browser, the connection is disconnected. The SSL / TLS module modifies the fields in the sent client_hello message so that the ja3 calculated by the business server is consistent with that of the browser, and the connection is restored.

[0043] This embodiment is another preferred implementation method. By using the traffic analysis module to identify sensitive data in plaintext traffic, sensitive traffic can be blocked in time to protect network security.

[0044] Example 4 The encryption and decryption flow control method based on transparent proxy includes the following steps: a. Use the transparent proxy module to determine whether the intranet port traffic and the external network port traffic are whitelisted IPs. If so, the decryption module is skipped directly to connect the intranet port traffic directly; otherwise, the traffic is transmitted to the decryption module; b. The decryption module establishes two connections to form a transfer station, one to the client and the other to the business server. The decryption module determines whether the traffic is SSL / TLS protocol; c. The plaintext traffic and the data decrypted by the decryption module are transmitted to the traffic analysis module. After being processed by the traffic analysis module, the processing results are stored in the shared memory. After the decryption module obtains the processing results, it restores the processing results to correspond to the two connections and releases or blocks them.

[0045] In the step a, the transparent proxy module is used to connect the traffic of the internal network port and the traffic of the external network port.

[0046] In the step a, the decryption module is used to decrypt the SSL / TLS payload data.

[0047] In the step b, judging whether the traffic is the SSL / TLS protocol through the decryption module means that when the traffic is judged to be the SSL / TLS protocol, the traffic session is marked, and the two connections enable the SSL / TLS module for encrypted communication; when the traffic is judged not to be the SSL / TLS protocol, the two connections use TCP communication.

[0048] The SSL / TLS module is used to build the SSL / TLS protocol and encrypt and decrypt messages.

[0049] In the step b, the connection to the business server simulates the browser fingerprint when client_hello occurs.

[0050] The simulated browser fingerprint means that after the business server receives the client_hello sent by the SSL / TLS module, the business server calculates ja3 based on the fields in the client_hello message and compares it with the browser's inherent ja3. If it is determined that it is not from the browser, the connection is disconnected. The SSL / TLS module modifies the fields in the sent client_hello message so that the ja3 calculated by the business server is consistent with that of the browser, and the connection is restored.

[0051] In step c, the traffic analysis module is used to identify sensitive data in plaintext traffic.

[0052] In the step c, the traffic analysis module is used to store the processing results in the shared memory.

[0053] In step c, the shared memory is used for the decryption module process and the traffic analysis module process to access together.

[0054] This embodiment is the best implementation method. It is based on an efficient event notification mechanism and a state machine to isolate the traffic of the internal network port and the external network port. The internal network port and the external network port do not interact directly. The internal network port and the external network port communicate indirectly through a decryption module, which can flexibly control network traffic.

[0055] Added whitelist function to release specified IP traffic, reduce calculation amount and improve efficiency.

[0056] Using modular design, unifying the interaction interfaces of each module and decoupling the code, it is possible to quickly replace modules and improve maintenance convenience.

[0057] The flow control method of the present invention is specifically described below: See also Figure 1 , the encryption and decryption flow control method based on transparent proxy includes the following steps: S1. Configure whitelist IP to ignore system processes and not perform traffic analysis; S2: Verify whether the IP address is in the whitelist based on the request and handshake packet sent by the traffic source. If it is, skip the following process. If it is not in the whitelist, verify whether the subsequent message has TLS protocol characteristics. If it meets the protocol characteristics, start step S3. Otherwise, go directly to step S4. S3, used to simulate browser fingerprint to establish TLS protocol communication, then obtain plaintext traffic and return it to the decryption module; S4. The traffic analysis module obtains the plaintext traffic from the decryption module, one is the original traffic of the non-TLS protocol, and the other is the traffic of the TLS protocol. The traffic decrypted by the SSL / TLS module will be verified. S5. The traffic analysis module stores the verification processing result in the shared memory, and the decryption module obtains the result from the shared memory and blocks or releases the communication according to the result.

[0058] The system structure of the present invention is shown in Figure 2 and Figure 3 : Transparent proxy module: used to take over the internal network port and the external network port, allowing the internal network port and the external network port traffic to communicate directly, or the decryption module can be connected in series to the process of the transparent proxy module. If it is in the whitelist, it can communicate directly, otherwise the decryption module is connected in series for processing.

[0059] Decryption module: After receiving the traffic passed by the transparent proxy module, it determines whether the traffic type is the SSL / TLS protocol. If so, the SSL / TLS module is enabled. If not, the message is directly handed over to the traffic analysis module for processing; after the SSL / TLS module processes, the message decrypted by SSL / TLS is obtained and then handed over to the traffic analysis module. At the same time, the decryption module will not block and wait for the response of the traffic analysis module, but will process other traffic.

[0060] SSL / TLS module: After receiving the traffic passed by the decryption module, it will build the browser's Client_hello message to connect to the destination server to prevent being identified as a crawler by the destination server.

[0061] Traffic analysis module: It is a process or another thread. As a separate execution process, it can prevent the entire network from being stuck due to time-consuming processing. When the traffic analysis module completes the processing, it is stored in the shared memory. As the producer, the decryption module, as the consumer, takes out the results as needed, and then releases the previous traffic or blocks the traffic according to the results.

Claims

1. The encryption and decryption flow control method based on transparent proxy is characterized by: The following steps are involved: a. Use the transparent proxy module to determine whether the intranet port traffic and the external network port traffic are whitelisted IPs. If so, the decryption module is skipped directly to connect the intranet port traffic directly; otherwise, the traffic is transmitted to the decryption module; b. The decryption module establishes two connections to form a transfer station, one to the client and the other to the business server. The decryption module determines whether the traffic is SSL / TLS protocol; c. The plaintext traffic and the data decrypted by the decryption module are transmitted to the traffic analysis module. After being processed by the traffic analysis module, the processing results are stored in the shared memory. After the decryption module obtains the processing results, it restores the processing results to correspond to the two connections and releases or blocks them.

2. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In the step a, the transparent proxy module is used to connect the traffic of the internal network port and the traffic of the external network port.

3. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In the step a, the decryption module is used to decrypt the SSL / TLS payload data.

4. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In the step b, judging whether the traffic is the SSL / TLS protocol through the decryption module means that when the traffic is judged to be the SSL / TLS protocol, the traffic session is marked, and the two connections enable the SSL / TLS module for encrypted communication; when the traffic is judged not to be the SSL / TLS protocol, the two connections use TCP communication.

5. The encryption and decryption flow control method based on transparent proxy according to claim 4 is characterized in that: The SSL / TLS module is used to build the SSL / TLS protocol and encrypt and decrypt messages.

6. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In the step b, the connection to the business server simulates the browser fingerprint when client_hello occurs.

7. The encryption and decryption flow control method based on transparent proxy according to claim 6 is characterized in that: The simulated browser fingerprint means that after the business server receives the client_hello sent by the SSL / TLS module, the business server calculates ja3 based on the fields in the client_hello message and compares it with the browser's inherent ja3. If it is determined that it is not from the browser, the connection is disconnected. The SSL / TLS module modifies the fields in the sent client_hello message so that the ja3 calculated by the business server is consistent with that of the browser, and the connection is restored.

8. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In step c, the traffic analysis module is used to identify sensitive data in plaintext traffic.

9. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In the step c, the traffic analysis module is used to store the processing results in the shared memory.

10. The encryption and decryption flow control method based on transparent proxy according to claim 1 is characterized in that: In step c, the shared memory is used for the decryption module process and the traffic analysis module process to access together.

Images