Transparent proxy-based API isolation gateway system and resource management method

Abstract

The application discloses a kind of API isolation gateway systems and resource management methods based on transparent proxy, belong to information security technical field.The gateway system includes front transfer module and rear transfer module, both are configured with transparent proxy function;Front transfer module is connected with client, for receiving the API request of client, and API request and its original connection information are sent to rear transfer module;Rear transfer module is used to establish connection according to the original connection information received with target server, and API request is forwarded;And response data returned by target server is sent to front transfer module;System management module is used to automatically identify, aggregate and carding API business according to log information, generates API interface information, and according to carding stage control the working mode of front transfer module and rear transfer module.The application solves the problem that API isolation gateway equipment needs to be manually carded API business and needs to be modified existing business network when deploying in prior art.

Description

Technical Field

[0001] This invention belongs to the field of information security technology, specifically relating to an API isolation gateway system and resource management method based on transparent proxy. Background Technology

[0002] With the widespread adoption of microservice architecture, Application Programming Interfaces (APIs) have become the core of data interaction between modern applications. To ensure the security of core business data, deploying API isolation gateway devices at critical network nodes has become a common security strategy. Existing API isolation gateway systems typically adopt an architecture of "front-end forwarding module + back-end forwarding module + dual unidirectional optical gateways / network gateways".

[0003] In this existing technical solution, before deploying the API isolation gateway device, all relevant API service interfaces in the network link need to be manually reviewed and pre-configured. Simultaneously, proxy IP addresses need to be configured on both the front-end and back-end forwarding modules, and clients are required to change their IP address when accessing the target server to the proxy IP address of the front-end forwarding module. Furthermore, if the server has a source IP-based authentication mechanism, corresponding modifications are necessary to ensure that the client IP proxied by the back-end forwarding module can pass authentication.

[0004] This existing technical solution has significant technical drawbacks: First, it is costly to deploy and has poor adaptability. When deploying a new API isolation gateway device in a complex network, it is necessary to comprehensively review and manually configure the API services in the relevant links beforehand. Automatic identification and review are not possible, resulting in a cumbersome and inefficient deployment process. Second, it is highly intrusive to existing services. The deployment process requires modifications to the service providers on both sides of the network, such as modifying client configurations and adjusting server authentication policies to adapt to the new network topology. This not only increases the difficulty of deployment but may also impact existing services. Summary of the Invention

[0005] The purpose of this invention is to provide an API isolation gateway system and resource management method based on transparent proxy, which can solve the problems in the prior art where API isolation gateway devices need to be manually sorted out for API services and existing business networks need to be modified when deployed.

[0006] In a first aspect, the present invention provides an API isolation gateway system based on transparent proxy, including a front-end forwarding module, a back-end forwarding module, a dual unidirectional isolation module connected between the front-end forwarding module and the back-end forwarding module, and a system management module connected to the front-end forwarding module and the back-end forwarding module respectively. Both the front-end forwarding module and the back-end forwarding module are configured with transparent proxy function to transparently send the received data packets from kernel mode to user mode program for processing. The front-end forwarding module is connected to the client and is used to receive API requests from the client. It then sends the API requests and the original connection information obtained through parsing to the back-end forwarding module through the dual unidirectional isolation module. The back-end forwarding module is used to establish a connection with the target server based on the received original connection information and forward API requests; and to receive the response data returned by the target server and send it to the front-end forwarding module through the dual unidirectional isolation module. The system management module includes an API business sorting submodule, which is used to automatically identify, aggregate, and sort the API business that passes through based on the log information reported by the front-end forwarding module or the back-end forwarding module, generate API interface information, and control the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

[0007] As a further implementation, the transparent proxy function is implemented through the Linux kernel's TProxy mechanism.

[0008] As a further implementation, the raw connection information includes the source IP address, source port, destination IP address, and destination port of the API request.

[0009] As a further implementation method, the working modes include: a fully autonomous sorting mode, which allows all API services that are not explicitly prohibited to pass through during the initial deployment phase, and logs and sorts all API services that pass through; a sorting and verification mode, which stops logging API services that have been sorted out after a certain period of sorting out, and only logs and sorts out API services that have not been sorted out; and a control mode, which allows only API services that have passed verification to pass through after sorting out, and blocks API services that have not passed verification or have not been sorted out.

[0010] As a further implementation, the system management module also includes a configuration management submodule, which is used to receive the user's selection and adjustment of the API interface information sorted out by the API business sorting submodule, and generate control policies based on the adjusted results and send them to the front-end forwarding module and the back-end forwarding module.

[0011] As a further implementation, both the pre-forwarding module and the post-forwarding module include: The protocol parsing unit is used to parse API requests using the HTTP or HTTPS protocol. The Certificate Management unit is used to manage certificates related to HTTPS link authentication.

[0012] As a further implementation, the dual unidirectional isolation module is a dual unidirectional optical gate or network gate, used to achieve physical isolation and unidirectional data transmission between the front-end forwarding module and the back-end forwarding module.

[0013] As a further implementation, the API business sorting submodule is used to automatically identify, aggregate, and sort the API business that passes through based on the log information reported by the front-end forwarding module and the back-end forwarding module, generate API interface information, and control the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

[0014] A second aspect of the present invention provides a resource management method for an API isolation gateway system based on a transparent proxy, comprising the following steps: By using a front-end forwarding module configured with transparent proxy functionality, API requests sent by clients are received and parsed to obtain the original connection information; The front-end forwarding module sends the API request and original connection information to the back-end forwarding module through the dual unidirectional isolation module; The post-forwarding module establishes a connection with the target server based on the original connection information and forwards the API request; The back-end forwarding module receives the response data returned by the target server and sends it to the front-end forwarding module through the dual one-way isolation module; The forwarding module returns the response data to the client; The system management module automatically identifies, aggregates, and sorts the API services that pass through based on the log information reported by the front-end forwarding module or the back-end forwarding module, generates API interface information, and controls the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

[0015] A third aspect of the present invention provides a network device including the API isolation gateway system based on a transparent proxy as described in the first aspect of the present invention.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention discloses an API isolation gateway system and resource management method based on transparent proxy. By configuring transparent proxy functionality on both the front-end and back-end forwarding modules and combining it with a phased API service self-management mechanism, the following beneficial effects are achieved: First, due to the use of transparent proxy technology, clients can directly access the target server's original IP address without being aware of the gateway device's existence, and the server also does not need to be aware of the gateway device, thus achieving "unobtrusive" deployment of the gateway device and greatly reducing the intrusion and workload of existing services. Second, by automatically identifying and aggregating passing API services through the system management module and adopting a phased mode of "fully autonomous management - management verification - control," not only is the management of API services automated, avoiding tedious and error-prone manual operations, but also, through phased switching, the integrity of the management is ensured while effectively controlling the amount of system logs, thus improving management efficiency. Therefore, this invention significantly reduces the deployment difficulty and cost of API isolation gateways in complex network environments, improves the accuracy and efficiency of API service management, and ultimately achieves precise and secure control over API traffic. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the architecture of an API isolation gateway system in the prior art; Figure 2 This is a schematic diagram of the architecture of an API isolation gateway system based on transparent proxy disclosed in an embodiment of the present invention; Figure 3 This is a flowchart disclosed in an embodiment of the present invention for extracting API business information by analyzing transparent proxy API log information; Figure 4 This is a flowchart of API business information aggregation disclosed in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0019] Example 1 like Figures 2 to 4As shown, this embodiment provides an API isolation gateway system based on transparent proxy, including a front-end forwarding module, a back-end forwarding module, a dual unidirectional isolation module connected between the front-end forwarding module and the back-end forwarding module, and a system management module connected to the front-end forwarding module and the back-end forwarding module respectively. Both the front-end forwarding module and the back-end forwarding module are configured with transparent proxy function, which is used to transparently send the received data packets from the kernel mode to the user mode program for processing. The front-end forwarding module is connected to the client and is used to receive API requests from the client. It then sends the API requests and the original connection information obtained through parsing to the back-end forwarding module through the dual unidirectional isolation module. The back-end forwarding module is used to establish a connection with the target server based on the received original connection information and forward API requests; and to receive the response data returned by the target server and send it to the front-end forwarding module through the dual unidirectional isolation module. The system management module includes an API business sorting submodule, which is used to automatically identify, aggregate, and sort the API business that passes through based on the log information reported by the front-end forwarding module or the back-end forwarding module, generate API interface information, and control the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

[0020] In this invention, the front-end forwarding module and the back-end forwarding module transmit "request data" and "response data" respectively through two unidirectional channels (e.g., two unidirectional optical gates or two different channels of a dual unidirectional optical gate). The two modules do not directly maintain the TCP connection state; instead, they use a "private protocol encapsulation" method to unidirectionally transmit the original connection information (such as four-tuples, parsed HTTP requests / responses) in message form. Upon receiving a request, the back-end forwarding module "establishes a new TCP connection with the target server." When the response is returned, the front-end forwarding module returns the response to the client based on its local cache mapping (such as the TCP connection established with the client), thereby avoiding real-time state synchronization across isolated modules.

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0022] Figure 1 This is a schematic diagram of the architecture of an existing API isolation gateway system. This existing technical solution has obvious technical defects.

[0023] To address the technical problems existing in the API isolation gateway system in the prior art, this embodiment provides an API isolation gateway system based on transparent proxy, which aims to solve the problems of difficult deployment of API isolation gateway devices and the need for manual sorting of API services in complex network environments.

[0024] like Figure 2As shown, the system mainly includes: a front-end forwarding module, a back-end forwarding module, a dual unidirectional optical gate / network gate module, and a system management module.

[0025] The front-end forwarding module is deployed on the client-side network, and the back-end forwarding module is deployed on the server-side network. Physical isolation and unidirectional data communication between the two are achieved through dual unidirectional optical gateways / network gateways. The system management module is connected to both the front-end and back-end forwarding modules for unified configuration and management.

[0026] Core working principles and implementation steps: 1. Initial Deployment and Transparent Proxy Configuration: First, after the API isolation gateway device starts up, select "Transparent Transmission Mode" through the system management module and enable "API Fully Autonomous Routing Function". The system management module will then issue corresponding instructions to the front-end forwarding module and the back-end forwarding module. Upon receiving the instructions, the user-space programs of these two modules automatically configure the underlying Linux kernel's TProxy transparent proxy function. This configuration ensures that any TCP packets entering the network interface card and destined for the server's real IP address can be transparently redirected by the kernel to the user-space program for processing, without requiring the client to modify its destination address.

[0027] 2. Transparent Processing and Log Generation of API Requests: When a client initiates an API request, the request data packet arrives at the front-end forwarding module. Since TProxy is enabled, after the Linux kernel completes a TCP three-way handshake with the client, it sends the request data (HTTP / HTTPS message) to the user-space program of the front-end forwarding module. Upon receiving the request, the user-space program first performs protocol parsing. If it is an HTTPS request, it completes a TLS handshake with the client through the built-in certificate management module and decrypts the request. Simultaneously, the program obtains the original four-tuple information of the request (source IP, source port, destination IP, destination port). Based on the parsed HTTP protocol information (such as URL, Host, Method, etc.) and the original four-tuple, the user-space program generates a "transparent proxy API log" and sends it to the log management function of the system management module. At the same time, the original four-tuple information and the API request data are encapsulated into a private protocol format and transmitted unidirectionally to the back-end forwarding module through a dual unidirectional optical gateway / network gateway module.

[0028] 3. API Request Forwarding and Server Response: After receiving the private protocol data, the back-end forwarding module unpacks it. Its user-space program, based on the parsed original four-tuple information (especially the source IP and source port), uses the `bind` method from socket programming to bind its own socket to the original client's IP and port, and then initiates a TCP connection to the target server. This crucial step allows the back-end forwarding module to establish a connection with the server as the original client. After the connection is established, the back-end forwarding module forwards the API request to the server. Upon receiving the request, the server, seeing the original client's IP as the source IP, is unaware of the API isolation gateway device's existence, directly processes the business logic, and returns the response data.

[0029] 4. Transparent Return of Response Data: The response data packet returned by the server is destined for the original client's IP address, and therefore passes through the back-end forwarding module. Since the back-end forwarding module also uses TProxy transparent proxy functionality, the Linux kernel transparently forwards this data packet, which is not destined for the local IP, to the user-space program of the back-end forwarding module. After receiving the response data, the user-space program parses it, generates corresponding logs, and sends them to the system management module. Simultaneously, it transmits the response data unidirectionally back to the front-end forwarding module via a private protocol and a dual unidirectional optical gateway / network gateway module. Upon receiving the response data, the front-end forwarding module returns the response data to the client through the previously established and locally stored socket channel. This completes a full API access process that is transparent to both the client and server.

[0030] 5. Phased self-management of API business: The API business management sub-module of the system management module is responsible for controlling the entire management process.

[0031] Phase 1: Fully Autonomous Review. In the initial phase, the system is in "fully autonomous review mode." During this phase, all API services not explicitly prohibited are allowed to pass through. The system has pre-set basic rejection rules (such as prohibiting access to management ports and non-HTTP traffic). The API service review submodule is triggered each time the forwarding module generates and reports a log. Its processing flow is as follows: a. Extraction: Extract API business information from the logs, including protocol type, service address, port, request URL path, request method, Host field, etc. Figure 3 As shown.

[0032] b. Aggregation: Before aggregation, the extracted log protocol type, service address, and port information are initially categorized using the K-Means clustering algorithm. The initially categorized logs are then aggregated periodically (e.g., daily at 00:00 and 12:00). For example, two logs with request paths / api / user / 123 and / api / user / 456 are aggregated into an intermediate API interface / api / user / {id} based on the path pattern. Simultaneously, the request methods of the two logs are aggregated; for example, the request method of the / api / user / 123 log is GET, and the request method of the / api / user / 456 log is also GET. After aggregation, the request methods of the intermediate API become {GET, GET}. After deduplicating all the log request methods aggregated into one intermediate API interface, a complete API interface is generated. Figure 4 The specification constraint algorithm generates the final business API rule interface.

[0033] c. Specification Control: During the aggregation process, control the number of API interfaces generated in the final stage. For example, aggregate to about 70% of the system specifications first, and reserve 30% of the resources for subsequent management functions. This stage continues until the system believes that 70%-80% of the business has been sorted out, or has been running for a predetermined period of time (such as half a month).

[0034] Phase Two: API Review and Verification. Once a predetermined threshold is reached, the administrator can switch to "API Review and Verification Mode" via the system management module. In this mode, the system management module will distribute the reviewed and aggregated API interface list to the front-end forwarding module. When processing requests subsequently, the front-end forwarding module will no longer log requests for APIs matching the reviewed interfaces; it will only log and continuously review the remaining 20%-30% of API requests that do not match. This phase effectively reduces the amount of logs and focuses on the discovery of new business applications.

[0035] Phase Three: Control Mode. Once the API business review coverage exceeds 95% or a preset time (e.g., three months) has elapsed, the administrator can switch the system to "API Control Mode." Before switching, the administrator can use the configuration management submodule of the system management module to conduct a final review and adjustment of the reviewed API interface list (e.g., manually or through AI analysis, marking untrusted API interfaces as "blocked"). After switching, the front-end forwarding module will, according to the final control policy, only allow trusted API requests to pass, blocking unreviewed or untrusted API requests and recording the blocking logs.

[0036] In the above embodiments, the transparent proxy function is implemented through the Linux kernel's TProxy mechanism. As an alternative, other network frameworks or technologies with similar functionality can be used, such as DPDK (Data Plane Development Kit) to directly process data packets in user space to achieve high-performance transparent proxying. Furthermore, regarding the aggregation algorithm for API business processing, in addition to rule-based path aggregation, machine learning algorithms can be introduced to analyze API request parameters, call sequences, and other behaviors, achieving more intelligent and accurate API interface identification and processing. Before switching to control mode, the "trustworthiness" judgment of API interfaces, besides manual selection and rule-based filtering, can also be automatically completed by the system using threat intelligence and traffic baseline analysis, further improving the level of automation.

[0037] Example 2 This embodiment provides a resource management method for an API isolation gateway system based on a transparent proxy. Based on the API isolation gateway system based on embodiment one, the method includes the following steps: By using a front-end forwarding module configured with transparent proxy functionality, API requests sent by clients are received and parsed to obtain the original connection information; The front-end forwarding module sends the API request and original connection information to the back-end forwarding module through the dual unidirectional isolation module; The post-forwarding module establishes a connection with the target server based on the original connection information and forwards the API request; The back-end forwarding module receives the response data returned by the target server and sends it to the front-end forwarding module through the dual one-way isolation module; The forwarding module returns the response data to the client; The system management module automatically identifies, aggregates, and sorts the API services that pass through based on the log information reported by the front-end forwarding module or the back-end forwarding module, generates API interface information, and controls the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

[0038] Example 3 This embodiment provides a network device, including the API isolation gateway system based on transparent proxy described in Embodiment 1 of the present invention.

[0039] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An API isolation gateway system based on transparent proxy, characterized in that, It includes a front-forwarding module, a back-forwarding module, a dual unidirectional isolation module connected between the front-forwarding module and the back-forwarding module, and a system management module connected to the front-forwarding module and the back-forwarding module respectively. Both the front-forwarding module and the back-forwarding module are configured with transparent proxy function, which is used to transparently send the received data packets from the kernel mode to the user mode program for processing. The front-end forwarding module is connected to the client and is used to receive API requests from the client and send the API requests and the original connection information obtained by parsing to the back-end forwarding module through the dual unidirectional isolation module. The post-forwarding module is used to establish a connection with the target server based on the received original connection information and forward the API request; and to receive the response data returned by the target server and send it to the front-forwarding module through the dual unidirectional isolation module. The system management module includes an API service sorting submodule, which is used to automatically identify, aggregate and sort the API services that pass through based on the log information reported by the front-end forwarding module or the back-end forwarding module, generate API interface information, and control the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

2. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, The transparent proxy function is implemented through the TProxy mechanism in the Linux kernel.

3. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, The original connection information includes the source IP address, source port, destination IP address, and destination port of the API request.

4. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, The operating modes include: a fully autonomous sorting mode, which allows all API services that are not explicitly prohibited to pass through during the initial deployment phase, and logs and sorts all API services that pass through; a sorting and verification mode, which stops logging API services that have been sorted out after a certain period of sorting out, and only logs and sorts out API services that have not been sorted out; and a control mode, which allows only verified API services to pass through after sorting out, and blocks API services that have not been verified or sorted out.

5. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, The system management module also includes a configuration management submodule, which is used to receive the user's selection and adjustment of the API interface information sorted out by the API business sorting submodule, and generate a control policy based on the adjusted result and send it to the front-end forwarding module and the back-end forwarding module.

6. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, Both the front-end forwarding module and the back-end forwarding module include: The protocol parsing unit is used to parse the API request using the HTTP or HTTPS protocol. The Certificate Management unit is used to manage certificates related to HTTPS link authentication.

7. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, The dual unidirectional isolation module is a dual unidirectional optical gate or network gate, used to achieve physical isolation and unidirectional data transmission between the front-end forwarding module and the rear-end forwarding module.

8. The API isolation gateway system based on transparent proxy as described in claim 1, characterized in that, The API service sorting submodule is used to automatically identify, aggregate, and sort the API services that have passed through based on the log information reported by the front-end forwarding module and the back-end forwarding module, generate API interface information, and control the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

9. A resource management method for an API isolation gateway system based on transparent proxy, characterized in that, An API isolation gateway system based on a transparent proxy as described in any one of claims 1-7 includes the following steps: By using a front-end forwarding module configured with transparent proxy functionality, API requests sent by clients are received and parsed to obtain the original connection information; The front-end forwarding module sends the API request and the original connection information to the back-end forwarding module through the dual one-way isolation module. The post-forwarding module establishes a connection with the target server based on the original connection information and forwards the API request; The back-forwarding module receives the response data returned by the target server and sends it to the front-forwarding module through the dual one-way isolation module; The forwarding module returns the response data to the client. The system management module automatically identifies, aggregates, and sorts the API services that pass through based on the log information reported by the front-end forwarding module or the back-end forwarding module, generates API interface information, and controls the working mode of the front-end forwarding module and the back-end forwarding module according to the sorting stage.

10. A network device, characterized in that, This includes an API isolation gateway system based on a transparent proxy as described in any one of claims 1-7.

Images