An API gateway service management method, system, and medium

Abstract

This invention relates to the field of distributed systems technology, specifically disclosing a method, system, and storage medium for managing API gateway services using the HTTP protocol. It addresses the problem of service discovery latency and inconsistency between node states in microservice architectures. The method includes: writing service information to a remote cache when the application service starts; configuring gateway tools and specifying request processing script paths; using scripts to preferentially retrieve the service list from local shared memory, and if empty, fetching the latest list from the remote cache; employing a load balancing algorithm to forward HTTP requests, automatically trying the next service if a failure occurs, and returning a degraded response if all services fail; executing a health check script to traverse the service list, and synchronously removing services with more than a threshold number of failures from both the remote cache and shared memory.

Description

Technical Field

[0001] This invention belongs to the field of microservice architecture technology, and in particular relates to an API gateway service management method, system and medium. Background Technology

[0002] In a microservice architecture, the API gateway, as the core hub for service calls, needs to dynamically manage the availability of backend services. Current mainstream solutions rely on a service registry (such as Consul or Eureka) to maintain the service list, with gateway nodes periodically pulling the service list from the registry and caching it locally. However, this mechanism has significant drawbacks: First, frequent access to the registry can lead to delays in service list updates due to network latency, especially in large distributed systems where real-time synchronization between the local cache and the service registry is difficult to guarantee. Second, when the gateway detects a faulty service through health checks, it typically only removes it from the local cache; however, other gateway nodes, unaware of this change, will still route requests to the faulty service, resulting in duplicate erroneous requests and wasted system resources. More seriously, the failure to update the faulty service in the global registry in a timely manner further exacerbates the problem of inconsistent service states across nodes, significantly reducing system availability and request success rate. Therefore, there is an urgent need for a management method that can efficiently obtain the real-time service list while ensuring the consistency of service states across all nodes in a distributed environment. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an API gateway service management method, system and medium that can efficiently obtain the real-time service list and ensure the consistency of service status of all nodes in a distributed environment, in order to address the shortcomings of the prior art.

[0004] The first aspect of this invention discloses an API gateway service management method, comprising the following steps:

[0005] After the application service starts, it writes the service information to the remote cache;

[0006] Configure the gateway tool and specify the script path for request processing;

[0007] The script retrieves the service list from local shared memory; if the list is empty, it retrieves the latest list from the remote cache.

[0008] A load balancing algorithm is used to select a service and forward HTTP requests. If a request fails, the next service is tried. If all services fail, a degraded response is returned.

[0009] Execute a health check script to traverse the service list and check availability. If the number of service failures exceeds the threshold, remove it from the remote cache and shared memory.

[0010] The load balancing algorithm described above includes a round-robin algorithm, a random selection algorithm, or a service weight-based algorithm.

[0011] The health check step in the above method includes calling the service's preset interface to detect the response status. If the response status is HTTP code 200, the service is determined to be available.

[0012] The above method also includes triggering health check steps based on scheduled tasks, and the interval of the scheduled tasks can be configured.

[0013] In the above method, the remote cache includes Redis or Memcached.

[0014] A second aspect of the present invention discloses an API gateway system, comprising:

[0015] The service registration module is used to write the service information of application services into a remote cache;

[0016] The gateway configuration module integrates OpenResty, configures the location, and specifies the script path;

[0017] The request processing module executes a script to obtain a list of services from local shared memory, implements a load balancing algorithm to forward requests, and returns a degraded response when all services fail.

[0018] The health check module executes scripts to retrieve a list of services from a remote cache, detects service availability, and dynamically updates the cache and shared memory.

[0019] The health check module in the above system also includes a failure counter. If the number of service failures exceeds a configurable threshold, the service will be automatically removed.

[0020] The system also includes a scheduled task module, which uses the crontab mechanism to set up scheduled health checks.

[0021] In the above system, the request processing module supports multiple load balancing algorithms and implements request forwarding logic through scripts.

[0022] A third aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed, implements the method described in the first aspect.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] 1. The service list is retrieved from local shared memory first via script; if the list is empty, the latest list is fetched from the remote cache. This constructs a two-layer structure of local shared memory (high-speed access) and remote cache (persistent storage), which optimizes the service list retrieval speed (avoiding frequent access to remote resources) and ensures data real-time performance through remote caching. Compared with traditional service discovery mechanisms that rely solely on a registry center or local cache (such as Eureka, which only periodically fetches from the registry center), this resolves the contradiction between response latency and data consistency.

[0025] 2. When the number of failed service health checks exceeds a threshold, the service is synchronously removed from the remote cache and shared memory. In traditional solutions, health checks typically only update the local cache or provide a one-way notification to the registry, allowing other gateway nodes to still access the faulty service. This invention ensures that all gateway nodes are aware of service status changes in real time by synchronously updating the remote cache (global state) and local shared memory (node ​​state), thus solving the problem of inconsistent service states in a distributed environment.

[0026] In summary, this invention combines a two-layer service list acquisition mechanism with a distributed consistency service removal mechanism. While ensuring efficient acquisition of the service list, it achieves real-time synchronization and distributed consistency of service status by collaboratively updating local shared memory and remote cache. Attached Figure Description

[0027] Figure 1 This is a schematic diagram illustrating the operating principle of the API gateway service of this invention.

[0028] Figure 2 This is a block diagram of the API gateway system of the present invention. Detailed Implementation

[0029] Terminology Explanation:

[0030] API (Application Programming Interface): A standardized interface for communication between services in a microservice architecture. Gateways manage HTTP protocol API requests to achieve unified entry point routing.

[0031] HTTP (Hypertext Transfer Protocol): The basic network protocol used by gateways to forward requests.

[0032] Redis: A high-performance key-value database used as a remote cache storage service list.

[0033] API Gateway.lua: API gateway script.

[0034] health_check.lua: Health check script.

[0035] Open Resty Service: OpenResty service (handles client requests, manages local service cache; fetches data from Redis when the local service list is not available).

[0036] HTTP Client: HTTP client.

[0037] Backend Service: Backend service.

[0038] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0039] Example 1

[0040] See Figure 1 As shown, an API gateway service management method includes the following steps:

[0041] After the application service starts, it writes the service information to the remote cache;

[0042] Configure the gateway tool and specify the script path for request processing;

[0043] The script retrieves the service list from local shared memory; if the list is empty, it retrieves the latest list from the remote cache.

[0044] A load balancing algorithm is used to select a service and forward HTTP requests. If a request fails, the next service is tried. If all services fail, a degraded response is returned.

[0045] Execute a health check script to traverse the service list and check availability. If the number of service failures exceeds the threshold, remove it from the remote cache and shared memory.

[0046] like Figure 1As shown, after the application service (Backend Service) starts, its service information (including the register IP and port, which come from the backend service) is written to the remote cache Redis through the service registration module, establishing a globally unified service list storage source. When a client (Http Client) initiates an HTTP request, the Open Resty Service, as the gateway core, receives the request and triggers the execution of the APIGateway.lua script. This script first reads the service list from local shared memory (such as OpenResty's lua_shared_dict); if the list is empty, it immediately pulls the latest service list from Redis and synchronously caches it in shared memory, significantly reducing remote access latency through the local caching mechanism. The script dynamically selects the target service according to a preset load balancing algorithm (such as round-robin) and forwards the request to the corresponding application service (Backend Service) through the internal Http Client component. If the request fails (e.g., the response HTTP status code is not 200), the script automatically switches to the next available service node; when all service nodes are unavailable, it returns a preset downgraded response (e.g., a JSON-formatted error message "Service is temporarily unavailable") to ensure the final fault tolerance of the request chain.

[0047] The health check process is driven by the `health_check.lua` script, which periodically retrieves a list of services from Redis and iterates through each service's preset health interface (PORT) and registration IP (Register IP). When a service responds with a status code of 200, the failure counter for that service is reset; if a check fails (e.g., connection timeout or a non-200 response), the failure count is incremented. Once the cumulative number of failures for a specific service exceeds a configurable threshold (e.g., 3 times), the script synchronously removes the failed service node from both Redis and local shared memory, ensuring that all gateway nodes share a consistent service status view in real time. This bidirectional removal mechanism completely solves the problem of inconsistent request routing caused by the asynchrony between local cache and registry status in traditional solutions, while also ensuring global real-time synchronization of fault information through Redis's persistence features.

[0048] In implementation, those skilled in the art can use open-source tools such as OpenResty (a Lua extension of Nginx) and Redis. First, when the application service starts, it automatically writes its IP address and port information to the Redis cache. For example, when an e-commerce application starts, it calls the registration API and adds "192.168.1.1:8080" to the Redis list. In the gateway configuration step, a location rule is added to the nginx.conf file of OpenResty, specifying that content_by_lua_file points to the path of the api-gateway.lua script, so that all requests are routed to this script for processing. The script logic is as follows: api-gateway.lua first retrieves the service list from OpenResty's shared memory (e.g., using lua_shared_dict); if the list is empty (e.g., at startup), it connects to Redis to pull the latest list and stores it in shared memory to accelerate subsequent access. Load balancing uses a round-robin algorithm (e.g., if the list contains services A, B, and C, A is selected sequentially to handle requests). If a request fails (e.g., an HTTP status code other than 200), the script automatically switches to B or C; if all fail, a degraded response is returned (e.g., a JSON message "No service available, please try again later"). Health checks are performed by the `health_check.lua` script, which periodically iterates through the service list: retrieving the list from Redis, checking each service (e.g., calling its ` / ping` interface), and resetting the failure count if the response status is HTTP 200; otherwise, incrementing the counter by 1, and removing the service from Redis and shared memory after 3 failures. For example, if a payment service has 3 consecutive no-ping responses, the script automatically removes it to prevent subsequent requests from being routed to that service. The entire process requires no manual intervention and is easy to deploy on a Linux server.

[0049] The above methods can shorten the request chain (reducing the intermediate layer compared to traditional nginx reverse proxy), improve overall stability (as long as there are available services in the cluster, it can provide services to the outside world); and avoid the accumulation of unavailable services by dynamically updating the service list.

[0050] In this embodiment, the load balancing algorithm includes a round-robin algorithm, a random selection algorithm, or an algorithm based on service weight.

[0051] By specifying the type of load balancing algorithm, such as round-robin (selecting sequentially), random (selecting randomly), or weight-based (distributing requests according to the weights configured in the service), flexibility can be enhanced to adapt to different business scenarios (e.g., high-traffic services can be assigned higher weights), resource utilization can be optimized, and the risk of single points of failure can be reduced.

[0052] In implementation, the algorithm logic is implemented in the api-gateway.lua script. There are several algorithms: **Round-Robin Algorithm:** A counter records the index of the previously selected service; subsequent requests select the service with the index incremented by 1 (e.g., if the service list is [A, B, C], the first request selects A, the second selects B). **Random Selection Algorithm:** A random number is generated to select the service (e.g., random number 1 selects A, random number 2 selects B). **Weight-Based Algorithm:** A weight is assigned to each service (e.g., service A has a weight of 50%, service B has a weight of 30%, and service C has a weight of 20%), and the script selects the service based on this weight ratio (e.g., random number 0-0.5 selects A, random number 0.5-0.8 selects B). Those skilled in the art can easily modify the script variables to configure the algorithm type without rewriting the code.

[0053] In this embodiment, the health check step includes calling the service's preset interface to detect the response status. If the response status is HTTP code 200, the service is determined to be available.

[0054] It should be noted that in the `health_check.lua` script, the detection interface for each service is defined as ` / ping` (other interfaces can be configured, such as ` / health`). When the script iterates through the service list, it sends an HTTP GET request (e.g., "http: / / 192.168.1.1:8080 / ping") to the service's IP:Port. If the response status code is 200, the service is marked as available and the failure count is reset. For example, the logistics service provides the ` / ping` interface, which the health check script calls; if the response is normal, the counter is reset to zero. If the interface response is slow or fails, the script logs the failure. Those skilled in the art only need to implement a simple ` / ping` handler on the server side. This standardized detection method ensures accurate and reliable health checks, avoids false positives, and improves system reliability.

[0055] In this embodiment, a health check step is also included based on a scheduled task, and the interval between scheduled tasks is configurable.

[0056] By implementing automated periodic checks, the service list can be ensured to be updated in real time; the interval can be configured to adapt to different environments (such as high-frequency checks for critical systems).

[0057] In implementation, on a Linux server, the crontab file is used to edit scheduled tasks. For example, adding " / 30 * * * * curl http: / / localhost / health_check" to the crontab file indicates that a health check script will be executed every 30 minutes. The curl command calls the access path of health_check.lua to trigger the check logic. The interval is configurable (e.g., changed to " / 10 * * * *" every 10 minutes), and technicians can easily adjust it by modifying the crontab entry without restarting the service. For example, in an e-commerce system, a check can be set to occur every 1 minute to ensure service availability during peak hours.

[0058] In this embodiment, the remote cache includes Redis or Memcached.

[0059] During implementation, choose either Redis or Memcached as the remote cache. Redis is more commonly used, utilizing its list data structure to store service information (e.g., adding services by calling the Redis LPUSH command at application startup). If Memcached is chosen, its key-value pair storage service list is used. Technical personnel only need to specify the cache type in the configuration file (e.g., setting the Redis connection URL), and the script will automatically adapt. For example, the startup script might read the configuration and determine whether to use the Redis or Memcached client library.

[0060] Example 2

[0061] See Figure 2 and Figure 1 As shown, an API gateway system includes:

[0062] The service registration module is used to write the service information of application services into a remote cache;

[0063] The gateway configuration module integrates openresty (a Lua extension platform based on Nginx, the core runtime tool of the gateway), configures the location and specifies the script path;

[0064] The request processing module executes a script to obtain a list of services from local shared memory, implements a load balancing algorithm to forward requests, and returns a degraded response when all services fail.

[0065] The health check module executes scripts to retrieve a list of services from a remote cache, detects service availability, and dynamically updates the cache and shared memory.

[0066] In implementation, the system is deployed on a server. The service registration module provides an API (such as a Java or Python library) for the application, which is called to write to Redis when the application starts (e.g., an e-commerce application calls the `register_service()` function). The gateway configuration module uses OpenResty, configuring location rules in the `nginx.conf` file (e.g., `location / {content_by_lua_file / path / api-gateway.lua;}`) to specify the script path. The request processing module runs `api-gateway.lua`: the script reads a list from shared memory (or queries Redis if empty), and uses a round-robin algorithm to forward requests to backend services (e.g., forwarding user query requests to the product service); if all services fail, a fallback page is returned. The health check module runs `health_check.lua`: the script periodically pulls a list from Redis, iterates through the services, and calls / pings the interfaces; if a service fails more than a threshold (e.g., 3 times), it is automatically removed from Redis and shared memory. For example, in an order system, the health check module runs every 5 minutes, removing unavailable payment services. Those skilled in the art can integrate these modules using standard tools without custom hardware.

[0067] In this embodiment, the health check module also includes a failure counter. If the number of service failures exceeds a configurable threshold, the service will be automatically removed.

[0068] The health check module adds a failure counter with a configurable threshold (e.g., set to 3), automatically removing services that exceed the threshold; it provides a fault tolerance mechanism to avoid accidental removal due to momentary failures; and the adjustable threshold adapts to different network environments, improving accuracy.

[0069] During implementation, the `health_check.lua` script maintains a failure count variable (initially 0) for each service. The counter increments by 1 for each failed check and resets to zero for each successful check. Thresholds are stored in a configuration file (e.g., `threshold=3`), and the script reads and compares the count: if the count exceeds the threshold, a removal operation is performed (the key-value pair is deleted from Redis). Technical personnel can modify the threshold by editing the configuration file (e.g., changing it to 5), without modifying the script code. For example, in a weather API gateway, setting the threshold to 4 times can reduce the impact of network fluctuations.

[0070] In this embodiment, a scheduled task module is also included, which sets up scheduled health checks based on the crontab mechanism.

[0071] It should be noted that the system can add a scheduled task module (such as crontab) to trigger health checks at regular intervals, enabling unattended operation and ensuring continuous list updates; the interval can be configured to optimize performance.

[0072] In implementation, the scheduled task module is implemented through the operating system's crontab (or a similar systemd timer). Tasks are added to the crontab file (e.g., " / 15 * * * * curl http: / / localhost / health_check") to execute a health check script every 15 minutes. Technicians can modify the time interval (e.g., " / 5 * * * *" every 5 minutes), and the system schedules it automatically. For example, in a social media gateway, a check can be set up every 10 minutes to promptly address service changes.

[0073] In this embodiment, the request processing module supports multiple load balancing algorithms and implements request forwarding logic through scripts.

[0074] It should be noted that the request processing module supports algorithms such as polling and randomization, and forwarding is implemented by scripts; this can enhance the module's flexibility and support diverse business needs; scripting simplifies maintenance.

[0075] During implementation, the algorithm selection parameters (e.g., `algorithm="round_robin"`) are defined in the `api-gateway.lua` script. The script applies different logic based on these parameters: index increments during round-robin; a random function is used during randomization. Forwarding logic: the script receives HTTP requests, parses them, and proxies them to the selected service (e.g., sending requests using Lua's socket library). Technical personnel specify the algorithm type in the configuration file, and the script loads it dynamically. For example, in a video streaming gateway, switching to a random algorithm for load balancing.

[0076] Example 3

[0077] A computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method described in Embodiment 1.

[0078] During implementation, the media storage script files (such as api-gateway.lua and health_check.lua) and configuration files are stored on storage media (such as hard drives or SSDs). Technicians insert the media into the server, install OpenResty and Redis, and then run programs to automatically execute the steps. For example, the Lua scripts and crontab configurations can be packaged into a Docker image; during deployment, loading the image will run the complete gateway logic.

[0079] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An API gateway service management method, characterized in that, Includes the following steps: After the application service starts, the service information is written to a remote cache, which is either Redis or Memcached, for persistent storage of the global service list. Configure the gateway tool, integrate OpenResty, and specify the Lua script path for request processing; The script first retrieves the service list from the local shared memory. If the list is empty, it immediately pulls the latest list from the remote cache and updates it synchronously to the local shared memory, so as to build a two-layer cache structure that ensures high-speed local access and real-time remote access. A load balancing algorithm is used to select a service and forward HTTP requests. If a request fails, the next service is tried. If all services fail, a degraded response is returned. The script executes a health check, periodically traversing the service list and calling the service's preset interface to check the response status. If the response status is HTTP code 200, the service is considered available. If the interface response is slow or fails, the script records the service failure. If the number of service failures exceeds the configurable threshold, the service is removed from the remote cache and shared memory, ensuring that all gateway nodes are aware of service status changes in real time.

2. The method as described in claim 1, characterized in that, The load balancing algorithm includes a round-robin algorithm, a random selection algorithm, or an algorithm based on service weight.

3. An API gateway system, characterized in that, include: The service registration module is used to write the service information of the application service into a remote cache, which is either Redis or Memcached, and is used to persistently store the global service list. The gateway configuration module integrates OpenResty, configures the location, and specifies the Lua script path; The request processing module executes a script to first obtain the service list from the local shared memory. If the list is empty, it immediately pulls the latest list from the remote cache and updates it synchronously to the local shared memory, so as to build a two-layer cache structure that ensures high-speed local access and real-time remote access. Implement a load balancing algorithm to forward requests and return a degraded response when all services fail; The health check module executes scripts to retrieve a list of services from a remote cache, detects service availability, and dynamically updates the cache and shared memory. When checking service availability, if the response status is HTTP code 200, the service is considered available. If the interface response is slow or fails, the script logs the service failure. The health check module also includes a failure counter. If the number of service failures exceeds a configurable threshold, the service will be automatically removed from the remote cache and shared memory. It also includes a scheduled task module, which uses the crontab mechanism to set up scheduled health checks.

4. The system as described in claim 3, characterized in that, The request processing module supports multiple load balancing algorithms and implements request forwarding logic through scripts.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-2.

Images