An application cluster global health state rapid detection method based on an Api gateway

Abstract

The application discloses an application cluster global health state rapid detection method based on an Api gateway, and specifically comprises the following steps: 101) a preprocessing step, 102) a request step, 103) a detection step, and 104) a result processing step. The application provides an application cluster global health state rapid detection method based on an Api gateway, which is simple in framework and rapid in response.

Description

Technical Field

[0001] This invention relates to the field of server-side technology, and more specifically, to a method for rapid detection of the overall health status of an application cluster based on an API gateway. Background Technology

[0002] Currently, typical internet servers generally adopt a microservice deployment architecture. In a data center (based on a cloud provider's availability zone, it can be divided into multiple logical data centers, and each logical data center can be called a data center), multiple application services will run together. Each application service will then deploy multiple instances to run in a cluster, which is referred to as an application cluster.

[0003] Typical application service implementation frameworks include Dubbo and ZooKeeper registry centers. An API gateway is also deployed within the logical data center as the main entry point, connecting to specific application clusters through the registry center and providing a unified API interface for access and detection.

[0004] To ensure high availability of data center services, it is usually necessary to perform runtime health checks on each application cluster or specific application service instance within the data center. Once an unrecoverable failure is detected in some clusters, the disaster recovery program needs to be automatically started to switch traffic to another backup data center.

[0005] Existing detection methods require the detection machine to directly call the specified application clusters within the data center via IP addresses and corresponding API interfaces. This necessitates that each application cluster located behind the gateway and on the internal network expose its public IP address and port. This not only wastes valuable IP address resources but also poses a significant security risk to the entire data center and adds more complexity to the detection logic of the detection machine. Summary of the Invention

[0006] This invention overcomes the shortcomings of existing technologies and provides a simple and fast-responding method for rapid detection of the overall health status of application clusters based on API gateways.

[0007] The technical solution of the present invention is as follows:

[0008] A rapid method for detecting the overall health status of an application cluster based on an API gateway includes a detection machine, an API gateway, a load balancer, a service registry, and an application cluster. The detection machine, load balancer, and application cluster are connected to the API network, and both the load balancer and application cluster register with the service registry and maintain connectivity. The specific steps are as follows:

[0009] 101) Preprocessing steps: Deploy multiple testing machines across the country, with each testing machine containing a testing agent program to simulate App requests to the server;

[0010] Multiple data centers are deployed across the country, each with multiple application clusters. Each application cluster provides a health check interface to the outside world, which internally simulates database read and write, cache read and write, thread pool access and service calls, and returns the execution result information. The health check interface is integrated into the corresponding call protocol to form a health check service. This service is automatically published to the service registry center, and the application cluster name and instance machine name where the service is located are attached during registration.

[0011] 102) Request steps: The detection machine sends a detection request to the API gateway, and the API gateway then pulls the list of service providers for all application clusters from the service registry through the load balancer.

[0012] 103) Detection steps: After receiving the detection signal, the API gateway selects a specific provider from the service provider list for remote invocation each time; that is, each time the detection machine performs a detection, it transmits the specific application cluster name or instance machine name parameter to be detected. After receiving it, the API gateway searches in the locally cached service provider list, matching the name, parameter, and application cluster name or machine name attached to the service provider to locate the specific provider; the API gateway performs a remote invocation on the matched provider and returns the health detection invocation result to the detection machine;

[0013] 104) Result processing steps: Based on the detection results, classify and process them; classification and processing include notifying manual processing and automatically performing disaster recovery switchover.

[0014] Furthermore, there are at least eight testing sites across the country, mainly located on public cloud services in Guangzhou, Shanghai, and Beijing.

[0015] Furthermore, the health check metrics include the overall health response status of the application interface, which is either true or false. True indicates health, while false indicates a fault. Internally, the health check will make judgments based on metrics related to database read / write, cache read / write, thread pool access, and service calls. If any of these metrics is abnormal, the overall health response status will be false.

[0016] Database read / write involves randomly selecting a database (db) from each database instance and then quickly inserting, querying, and deleting a record in one of its built-in public tables. If all operations are successful, the test passes; otherwise, the database read / write operation fails.

[0017] Cache read / write: Perform an address map read / write operation on the Redis remote cache service connected to the application cluster. If successful, the check passes; otherwise, it fails.

[0018] Thread pool access is determined to be healthy by detecting whether the number of currently active threads in Tomcat is greater than a preset threshold.

[0019] Service calls are initiated because the API gateway itself uses a service call protocol to invoke the detection service interface within the cluster. If the detection service interface call is successful, it proves that the service call is successful; otherwise, it fails.

[0020] Furthermore, when the detection results from the detection machine are stored in the data collection data center, if more than 70% of the agents in a cluster report errors, it is considered that the cluster has a problem and the backend service needs to be switched to the corresponding disaster recovery cluster.

[0021] Furthermore, the testing machine performs tests at regular intervals, with the testing interval being no more than 15 seconds.

[0022] Furthermore, when automatically performing disaster recovery switchover, it also includes a recovery step: after the disaster recovery switchover procedure takes place, a switchover flag is set and relevant personnel are notified to manually switch back during off-peak hours.

[0023] Furthermore, the health detection interface in step 101) is integrated into the corresponding calling protocol, which is the Dubbo protocol.

[0024] The advantages of this invention compared to the prior art are:

[0025] This invention adds an API gateway to the data center and reduces other gateways. The added API gateway performs unified proxying. It only needs to specify the application cluster name or service instance name to forward the detection request to the target cluster or machine and return the detection result to the detection machine through the API gateway.

[0026] Because this invention adds an intermediate proxy layer, application clusters located on the intranet do not need to expose public IP addresses and ports, thus ensuring security; the detection machine only needs to interact with a separate API gateway, which also reduces complexity. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the present invention. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, wherein the same or similar reference numerals denote the same or similar elements or elements with similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as limiting the present invention.

[0029] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0030] The numbering of steps mentioned in the various embodiments is merely for descriptive convenience and does not imply a sequential relationship. Different steps in various specific embodiments can be combined in different orders to achieve the inventive objective of this invention.

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0032] like Figure 1 As shown, a rapid method for detecting the overall health status of an application cluster based on an API gateway includes a detection machine, an API gateway, a load balancer, a service registry, and an application cluster. The detection machine, load balancer, and application cluster are connected to the API network. Both the load balancer and application cluster register with the service registry and maintain connectivity. The detection machine connects to the application cluster through the API gateway and load balancer. The specific steps are as follows:

[0033] 101) Preprocessing steps: Multiple testing machines are deployed across the country, each with a testing agent program to simulate App requests to the server. There are at least 8 testing points nationwide, mainly distributed on public cloud services in Guangzhou, Shanghai, and Beijing.

[0034] Multiple data centers are deployed across the country. Each data center houses multiple application clusters, and each application cluster provides a health check interface to the outside world. This interface internally simulates database read / write operations, cache read / write operations, thread pool access, and service calls, and then returns the execution results, thus implementing the internal health check process. The health check interface is integrated into the corresponding calling protocol (the Dubbo protocol) to form a health check service. This service is automatically published to the service registry, and the application cluster name and instance machine name of the service are included during registration.

[0035] Specifically, the corresponding calling protocol defines a Java Dubbo interface as follows:

[0036]

[0037]

[0038]

[0039] Then, the application service has a built-in health check logic class. This class implements the above interface and is marked as a Dubbo protocol service. After the framework starts, it will automatically register it as a Dubbo service with the service center. A specific example is shown below:

[0040] @Service(interfaceClass=IServerCheckAction.class, version="1.0.0")

[0041] public class ServerCheckAction implements IServerCheckAction{

[0042] / / Specific health testing logic

[0043] }

[0044] Health checks include the overall health response status of the application interface, which is either true or false. True indicates health, and false indicates a fault. Internally, health checks assess several aspects, including database read / write operations, cache read / write operations, thread pool access, and service calls. If any of these indicators is abnormal, the overall health response status is set to false. A specific example is shown in the following code:

[0045]

[0046]

[0047] The testing process for each sub-indicator is as follows:

[0048] Database read / write operations randomly select a database (db) from each database instance and then test the insertion, query, and deletion of a record in one of its built-in public tables. If all operations are successful, the test passes; otherwise, the database read / write operation fails. A specific example is as follows:

[0049]

[0050]

[0051]

[0052] For cache read / write, perform an address map read / write operation on the remote dictionary service Redis connected to the application cluster. If successful, the operation passes; otherwise, it fails.

[0053] Thread pool access health is determined by checking whether the number of currently active threads in Tomcat exceeds a preset threshold; a specific example is as follows:

[0054]

[0055]

[0056] The number of Tomcat threads currently running is obtained through the JMX (Java Management Extensions) protocol.

[0057] Service calls are initiated because the API gateway itself uses a service call protocol to invoke the detection service interface within the cluster. If the detection service interface call is successful, it proves that the service call is successful; otherwise, it fails.

[0058] 102) Request steps: The detection machine sends a detection request to the API gateway, and the API gateway then pulls the list of service providers for all application clusters from the service registry through the load balancer.

[0059] 103) Detection Steps: After receiving the detection signal, the API gateway selects a specific provider from the service provider list for remote invocation each time. This solution takes integration with the Dubbo protocol as an example. The Dubbo protocol can provide a load balancer for selecting a specific service provider and supports application-layer custom load balancers for expansion, achieving good compatibility and scalability.

[0060] Each time a specific testing machine performs a test, it transmits the specific application cluster name or instance machine name parameter to be tested. After receiving the parameter, the API gateway searches for it in the locally cached list of service providers, matching the name, parameter, and application cluster name or machine name attached to the service provider to locate the specific provider. The API gateway then makes a remote call to the matched provider and returns the health check result to the testing machine.

[0061] 104) Result Processing Steps: Based on the detection results, classification and processing are performed. Classification and processing include notifying manual processing and automatic disaster recovery switchover. Specifically, if the detection results from the detection machine are present in the data collection data center, and more than 70% of the agents in a cluster report errors, then this cluster is considered to have a problem and the backend service needs to be switched to the corresponding disaster recovery cluster. The detection machine performs detections at regular intervals, typically within 15 seconds.

[0062] As a preferred option, when automatically performing disaster recovery switching, the following steps are also included: 105) Recovery steps: After the disaster recovery switching procedure takes place, a switching flag is set and relevant personnel are notified to manually switch back during off-peak hours.

[0063] Taking disaster recovery failover as an example, to simulate actual user network conditions, multiple proxies ensure reliable feedback. The failover speed is far superior to manual switching, minimizing user downtime. The data collection and failover processes are independent and do not interfere with each other, improving system reliability. Detecting errors uploaded by proxies helps in quickly analyzing the causes of cluster failures. The specific steps are as follows:

[0064] 201) Request Steps: The detection machine starts the detection agent program, first obtaining information about each data center cluster; after reading the cluster configuration information, it begins to check the health status of each cluster, completing a check of all clusters at regular intervals. Each cluster sequentially calls the check_health interface in the health detection service in a set order. The regular intervals are generally within 15 seconds.

[0065] 202) Voting steps: The detection agent program detects the actual behavior of the agent program in performing read and write operations on the business by calling the check_health interface on the server side; when an error occurs, in addition to information feedback, the detection agent program will also upload the error result for analysis of the cause;

[0066] Feedback from the detection agent program is stored in the data center's database. The switching program periodically checks the voting results. If more than 70% of the agents in a cluster report errors, the cluster is considered to have a problem and the backend service needs to be switched to the disaster recovery cluster on the other end. The switching program includes a failover prevention function: if fewer than three erroneous detection agents are uploaded, an alert is issued but no switch is performed; that is, the expected number of agents is n, and n is greater than 5. When the actual number of reported agents is insufficient, a network problem is assumed, and a manual disaster recovery switchover suggestion is issued. Periodic checks are performed every 5 seconds or less to expedite problem identification and minimize business impact.

[0067] Each detection machine first collects information from all clusters, then loops through the clusters, checking each cluster in turn. For each cluster, multiple addresses are used, and one address is obtained by taking the remainder after the current number of checks. The check results are reported to the data collection center. If all cluster checks are successful, the machine waits for an appropriate time according to the agreed-upon time for the entire process.

[0068] 203) Switching Procedure: Based on the voting results, if any cluster experiences upload errors exceeding the threshold, the disaster recovery switchover procedure automatically performs traffic switching. The collection and switching procedures are independent of each other and will not interfere with each other, thereby improving system reliability.

[0069] 204) Recovery Steps: After the disaster recovery switchover process occurs, a switchover flag is set and relevant personnel are notified. Generally, a manual switchback is performed during off-peak hours, but it can also be done manually during peak hours as needed, depending on the specific circumstances. The specific disaster recovery switchover method used is ClusterGuard.

[0070]

[0071]

[0072] The specific detection agent program calls the ClusterAgent method to perform cluster detection and assessment. It first obtains the configurations of all clusters from the data center, then loops through the clusters, checking each one sequentially. For each cluster, multiple addresses are used, and a single address is obtained by taking the remainder after the current number of checks. The results are reported to the detection machine, and error logs are also uploaded if errors are found. If all clusters are functioning correctly, the process waits for an agreed-upon time interval.

[0073]

[0074]

[0075]

[0076] The data center obtains the configuration information of all clusters, resulting in a JSON object similar to {"Cluster A":["https: / / domain1:port1 / slh / checkCluster.do?kind=checkHealth&serverApplicationName=stsServer","http: / / ip2:port2 / slh / checkCluster.do?kind=checkHealth&serverApplicationName=stsServer"],"Cluster B":["https: / / domain1:port1 / slh / check.do?svc=checkHealth&db=true&redis=true&thread=true","https: / / ip2:port2 / slh / check.do?svc=checkHealth&db=true&redis=true&thread=true","https: / / ip3:port3 / slh / check.do?svc=checkHealth&db=true&redis=true&thread=true"],...}. Each cluster has multiple access addresses to prevent false alarms due to a single line failure.

[0077] `serverApplicationName=stsServer` indicates that Dubbo calls need to be checked; `db=true` indicates that database read / write needs to be checked; `redis=true` indicates that cache read / write needs to be checked; `thread=true` indicates that thread limits need to be checked.

[0078] In summary, this solution adds an API gateway to the data center and reduces the number of other gateways. The added API gateway acts as a unified proxy, requiring only the application cluster name or service instance name to forward detection requests to the target cluster or machine, and then uniformly returns the detection results to the detection machine via the API gateway. Because of the added intermediate proxy layer, application clusters located on the internal network do not need to expose public IP addresses and ports, ensuring security; the detection machine only needs to interact with the separate API gateway, reducing complexity.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the concept of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for rapid detection of the overall health status of an application cluster based on an API gateway, characterized in that, This includes a testing machine, an API gateway, a load balancer, a service registry, and an application cluster. The testing machine, load balancer, and application cluster are connected to the API network. Both the load balancer and application cluster register with the service registry and maintain connectivity. The specific steps are as follows: 101) Preprocessing steps: Deploy multiple testing machines across the country, with each testing machine containing a testing agent program to simulate App requests to the server; Multiple data centers are deployed across the country, each with multiple application clusters. Each application cluster provides a health check interface to the outside world, which internally simulates database read and write, cache read and write, thread pool access and service calls, and returns the execution result information. The health check interface is integrated into the corresponding call protocol to form a health check service. This service is automatically published to the service registry center, and the application cluster name and instance machine name where the service is located are attached during registration. 102) Request steps: The detection machine sends a detection request to the API gateway, and the API gateway then pulls the list of service providers for all application clusters from the service registry through the load balancer. 103) Detection steps: After receiving the detection signal, the API gateway selects a specific provider from the service provider list for remote invocation each time; that is, the detection machine will transmit the specific application cluster name or instance machine name parameter to be detected each time it detects. After receiving it, the API gateway searches in the locally cached service provider list and matches the name, parameter and the application cluster name or machine name attached to the service provider to locate the specific provider. The API gateway makes remote calls to the matched providers and returns the health check results to the detection machine. 104) Result processing steps: Classify and process the results according to the test results; The categorized processing includes notifying manual handling and automatically performing disaster recovery switchover.

2. The method for rapid detection of the overall health status of an application cluster based on an API gateway according to claim 1, characterized in that: There are at least eight testing sites across the country, mainly located on public cloud services in Guangzhou, Shanghai, and Beijing.

3. The method for rapid detection of the overall health status of an application cluster based on an API gateway according to claim 1, characterized in that: The health check metrics include the overall health response status of the application interface, which is either true or false. True indicates health and false indicates a fault. Internally, the health check will judge based on metrics related to database read / write, cache read / write, thread pool access, and service calls. If any of these metrics is abnormal, the overall health response status will be false. Database read / write involves randomly selecting a database (db) from each database instance and then quickly inserting, querying, and deleting a record in one of its built-in public tables. If all operations are successful, the test passes; otherwise, the database read / write operation fails. Cache read / write: Perform an address map read / write operation on the Redis remote cache service connected to the application cluster. If successful, the check passes; otherwise, it fails. Thread pool access is determined to be healthy by detecting whether the number of currently active threads in Tomcat is greater than a preset threshold. Service calls are initiated because the API gateway itself uses a service call protocol to invoke the detection service interface within the cluster. If the detection service interface call is successful, it proves that the service call is successful; otherwise, it fails.

4. The method for rapid detection of the overall health status of an application cluster based on an API gateway according to claim 1, characterized in that: When the detection results from the detection machine are stored in the data collection data center, if more than 70% of the agents in a cluster report errors, it is considered that the cluster has a problem and the backend service needs to be switched to the corresponding disaster recovery cluster.

5. The method for rapid detection of the overall health status of an application cluster based on an API gateway according to claim 4, characterized in that: The testing machine performs tests at regular intervals, with the interval being no more than 15 seconds.

6. The method for rapid detection of the overall health status of an application cluster based on an API gateway according to claim 1, characterized in that: When automatically performing disaster recovery switchover, it also includes 105) recovery steps: after the disaster recovery switchover procedure takes place, a switchover flag is set and relevant personnel are notified to manually switch back during off-peak hours.

7. The method for rapid detection of the overall health status of an application cluster based on an API gateway according to claim 1, characterized in that: The health detection interface in step 101) is integrated into the corresponding calling protocol, which is the Dubbo protocol.

Images