Dynamic routing and traffic control method and system based on apisix
By integrating a service instance collector and a dynamic weight calculator into the APISIX gateway, the routing weight and rate limiting threshold are dynamically adjusted, solving the problem of uneven traffic distribution in the API gateway in a microservice architecture. This enables intelligent traffic control and security management, improving system stability and operational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INSPUR TIANYUAN COMM INFORMATION SYST CO LTD
- Filing Date
- 2026-03-05
- Publication Date
- 2026-07-21
AI Technical Summary
Existing API gateways lack the ability to dynamically perceive and intelligently schedule resource service capabilities in microservice architectures, resulting in uneven traffic distribution, high security and operational complexity, and difficulty in achieving real-time access control and call monitoring.
The service instance collector continuously collects operational metrics and stores them in a time-series database. The dynamic weight calculator calculates service capability scores, and the routing weights and rate limiting thresholds are adjusted based on the scores. This enhances the load balancing and traffic control plugins of the APISIX gateway, enabling multi-dimensional traffic management.
It provides access control and logging of resource capabilities, improves system stability and fault tolerance, enables refined operation and high real-time traffic scheduling, and adapts to the dynamic changes of the cloud-native environment.
Smart Images
Figure CN122437802A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of microservice architecture and API management technology, specifically a dynamic routing and traffic control method and system based on APISIX. Background Technology
[0002] The resource management system adopts a modern microservice architecture. The API gateway, as the entry point for all traffic, undertakes crucial responsibilities such as routing, load balancing, traffic control, and circuit breaking / degradation. Traditional routing and load balancing strategies (such as round-robin, least connections, and consistent hashing) are primarily based on simple network metrics or static configurations, and have the following drawbacks: (1) In the resource management system, a microservice architecture is adopted, and the application is broken down into many fine-grained services. As the entry point for all traffic, the API gateway's routing and traffic control capabilities become crucial. The system lacks monitoring, access control, and traffic control for API calls to resource service capabilities. Sharing capabilities directly exposes the original capability addresses, making it impossible to uniformly control access and record call logs, which poses security risks. (2) Existing traditional API gateways (such as Kong, Traefik) or earlier versions typically have the following problems in their configuration: Static configuration: Routing rules, upstream node lists, and traffic control policies (such as rate limiting and circuit breaking) mostly rely on configuration files. When backend service instances undergo dynamic changes (such as elastic scaling and fault migration), manual intervention or cumbersome API calls are required to reload the configuration, which cannot achieve true real-time performance and automation.
[0003] Lack of granularity and context-awareness: Traditional traffic control is usually based on simple dimensions, such as IP address or API key. They are difficult to dynamically adjust based on the content of the request (such as a specific header, user role in a JWT token), real-time system load, or external business rules.
[0004] Insufficient capability awareness: The gateway cannot proactively perceive the real-time health status, current load (such as CPU and memory usage) or business capabilities (such as whether it supports a certain function) of the backend service instances, resulting in uneven traffic distribution and an inability to intelligently route requests to the most suitable service instances.
[0005] High operational complexity: The aforementioned static and unobservable mechanisms mean that in dynamic cloud-native environments (such as Kubernetes), operations personnel need to maintain complex automated scripts to synchronize service status and gateway configurations, increasing system complexity and failure risks.
[0006] Apache APISIX is a dynamic, real-time, high-performance cloud-native API gateway built on top of Nginx and etcd. It provides powerful traffic management capabilities and is a crucial component in modern microservice architectures and cloud-native environments. However, the native APISIX primarily relies on its pre-defined routing and plugin configurations, lacking built-in capabilities for deep integration with external systems or dynamic routing and traffic control based on complex business logic. Therefore, there is an urgent need in the field for a technical solution that can perceive service capabilities in real time and implement intelligent, dynamic, and adaptive routing and traffic control accordingly. Summary of the Invention
[0007] The technical objective of this invention is to address the above-mentioned shortcomings by providing a dynamic routing and traffic control method and system based on APISIX, thereby building a more intelligent and adaptive dynamic control layer based on APISIX and improving the system's ability to manage resources.
[0008] In a first aspect, the present invention provides a dynamic routing and traffic control method based on APISIX, comprising the following steps: S1: Continuously collect the operational metrics of each service instance through the service instance collector and store them in the time series database; S2: Periodically query the operating metrics within a specified time window from the time series database using a dynamic weight calculator, calculate the service capability score for each service instance based on a weighted formula, and determine the health status level of the corresponding service instance based on the service capability score; dynamically adjust the routing weight and rate limiting threshold of the corresponding service instance based on the service capability score and health status level, and push the results to the APISIX gateway. S3: When a client request reaches the APISIX gateway, the enhanced load balancer selects a service instance based on real-time routing weights, while the traffic control plugin implements multi-dimensional traffic control based on the rate limiting threshold.
[0009] In S1, the operating metrics include at least one of the following: CPU utilization, memory utilization, request response latency, and HTTP error code ratio.
[0010] In S2, the service capability score of each service instance is calculated based on a weighted formula, including: service capability score S=w1×f1+w2×f2+w3×f3+w4×f4+..., where f1-f4 are normalization functions that map different dimensional operating indicators to a unified score range, and w1-w4 are weight coefficients that are dynamically adjusted according to the service type.
[0011] In S2, the health status levels include four levels: healthy, sub-healthy, dangerous, and offline.
[0012] In S2, pushing to the APISIX gateway includes: pushing the route weight and rate limiting threshold to the APISIX gateway in real time via the APISIX Control API, without restarting the gateway.
[0013] In S3, the traffic control plugin implements multi-dimensional traffic control based on the rate limiting threshold, including: the traffic control plugin implements multi-dimensional traffic control based on IP address, user identity, service type and API endpoint, and dynamically adjusts the rate limiting threshold according to the overall system load, business priority and user level.
[0014] Before step S1, the resource service capabilities that need to be shared are registered in the service registry, and the service access address and related parameters are configured; the registration process does not require modification of the backend service capability code.
[0015] When the error rate of a service instance continues to exceed a preset threshold, it is automatically marked as "unhealthy" and temporarily removed from the upstream service list to achieve circuit breaker protection. For service instances that recover after being circuit broken, they are reintroduced with the initial weight for gradual traffic recovery.
[0016] After S3 is executed, S1 is executed again.
[0017] Secondly, the present invention provides a dynamic routing and traffic control system based on APISIX, comprising: Metrics collection module: Used to continuously collect operational metrics of each service instance through the service instance collector and store them in the time series database; The strategy calculation and distribution module is used to periodically query the operating indicators within a specified time window from the time series database through a dynamic weight calculator, calculate the service capability score of each service instance based on the weighted formula, and determine the health status level of the corresponding service instance based on the service capability score; based on the service capability score and health status level, dynamically adjust the routing weight and rate limiting threshold of the corresponding service instance, and push it to the APISIX gateway. Routing and flow control module: When a client request arrives at the APISIX gateway, the enhanced load balancer selects a service instance based on real-time routing weights, while the flow control plugin implements multi-dimensional flow control based on flow limiting thresholds.
[0018] The APISIX-based dynamic routing and traffic control method and system of this invention have the following advantages: They provide access control for resource capabilities: based on the configured permission token, access to capabilities without the required permission token is denied, enhancing security; they provide logging for resource capability access: through shared access capabilities, call logs are automatically recorded according to the configuration, facilitating statistical analysis; they provide traffic control for resource capabilities: through shared access capabilities, based on the traffic control rules configured for the capabilities, traffic control is applied to external access, maintaining the stability of resource capabilities; they enhance system resilience: by quickly isolating or demoting unhealthy instances from the service pool, the spread of localized failures is effectively prevented, enhancing the fault tolerance of the entire microservice architecture; they provide refined operation: multi-dimensional monitoring data and dynamic adjustment capabilities provide operations personnel with more refined traffic management tools; they offer extremely high real-time performance and dynamism: fully utilizing APISIX's dependency-free restart dynamic capabilities, online routing and control policies can be modified without restarting, adapting to dynamic service changes in a cloud-native environment. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0020] The invention will be further described below with reference to the accompanying drawings.
[0021] Figure 1 The flowchart is a dynamic routing and flow control method based on APISIX according to Embodiment 1 of the present invention; Figure 2 This is an internal logic flowchart of the dynamic weight calculator according to Embodiment 1 of the present invention; Figure 3 This is a logical structure block diagram of an APISIX-based dynamic routing and flow control system according to Embodiment 2 of the present invention; Figure 4 This is an architecture design diagram of a dynamic routing and traffic control system based on APISIX according to Embodiment 2 of the present invention. Detailed Implementation
[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0023] It should be understood that in the description of the embodiments of the present invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. In the embodiments of the present invention, "multiple" refers to two or more.
[0024] In this embodiment of the invention, "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or both A and B existing simultaneously. Furthermore, in this document, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0025] Definitions: Apache APISIX is a high-performance, scalable cloud-native API gateway that acts as a traffic ingress controller in microservice architectures, providing unified API management, security protection, traffic governance, and observability capabilities. Based on the Nginx+LuaJIT technology stack, it adopts a dynamic, hot-reload design philosophy and is a top-level project (TLP) of the Apache Software Foundation.
[0026] etcd is a distributed, highly available, and strongly consistent key-value store designed for configuration management, service discovery, and coordination tasks in distributed systems. Example
[0027] like Figure 1 and Figure 2 As shown, the APISIX-based dynamic routing and traffic control method provided in this embodiment implements dynamic and intelligent routing and traffic control based on the Apache APISIX API gateway. It is a design and implementation method related to capability sharing and monitoring in a resource capability platform. By collecting performance metrics of backend service instances in real time and automatically, and dynamically and intelligently adjusting APISIX routing policies and traffic control parameters based on these real-time metrics, it achieves precise traffic scheduling, routing requests to configured service instances, thereby performing dynamic routing and improving the availability, stability, and resource utilization of the entire system. It is mainly used to solve problems such as dynamic routing, access control, call monitoring, and traffic control of resource service capabilities in resource management systems.
[0028] This embodiment provides a dynamic routing and traffic control method based on APISIX, which may include the following steps: S1: Continuously collect operational metrics for each service instance through the service instance collector and store them in the time-series database. Operational metrics include at least one of the following: CPU utilization, memory utilization, request response latency, and HTTP error code ratio.
[0029] This step involves data collection. The service instance collector continuously collects metrics from each instance and reports them.
[0030] The service instance collector can use tools such as Prometheus Exporter to obtain the aforementioned operational metrics in real time from the monitoring interfaces of each service instance through active pulling or passive pushing. The collection frequency can be configured according to actual needs, for example, set to collect once every 10 seconds to ensure data timeliness. The collected operational metrics will be stored in time-series databases such as InfluxDB and TimescaleDB in time-series format, along with metadata such as the unique identifier of the service instance, metric name, and collection timestamp, for subsequent metric queries.
[0031] S2: A dynamic weighted calculator periodically queries the time-series database for operational metrics within a specified time window. Based on a weighted formula, it calculates the service capability score for each service instance and determines its health status level. Based on the service capability score and health status level, it dynamically adjusts the routing weight and rate limiting threshold for the corresponding service instance and pushes this information to the APISIX gateway. The health status levels include: Healthy, Sub-healthy, Dangerous, and Offline.
[0032] The service capability score S = w1×f1 + w2×f2 + w3×f3 + w4×f4, where f1-f4 are normalization functions that map different dimensional operating indicators to a unified score range, and w1-w4 are weighting coefficients that are dynamically adjusted according to the service type.
[0033] This step involves metric analysis and aggregation. It periodically queries the time-series database for metrics of each service instance within a specified time window (e.g., the most recent minute) and calculates their health score.
[0034] The Dynamic Weight Calculator is an independent central computing service whose functions include indicator aggregation, service capability score calculation, health status assessment, and decision release.
[0035] Metrics aggregation: Receives runtime metric data from collectors of all service instances.
[0036] Service Capability Score Calculation: A real-time "Service Capability Score" is calculated for each service instance. This score is a comprehensive numerical value, and the calculation model can be designed as a weighted formula, for example: Service Capability Score S = w1×f1(CPU) + w2×f2(Memory) + w3×f3(ResponseTime) + w4×f4(ErrorRate) + ... , where f1, f2... are normalization functions used to map different units of performance metrics to a unified score range (e.g., 0-100); w1, w2... are weighting coefficients, which can be adjusted according to the service type (e.g., giving higher weight to CPU for CPU-intensive services). f1 is the normalization function for CPU utilization, f2 is the normalization function for memory utilization, f3 is the normalization function for request-response latency, and f4 is the normalization function for HTTP error code ratio.
[0037] Health status assessment: Determine the status of service instances based on service capability scores (e.g., healthy, sub-healthy, dangerous, offline).
[0038] Deployment Decision: The service capability score or calculated weight (for load balancing) of each service instance, along with the recommended rate limiting threshold, is pushed to the APISIX gateway. This is achieved by calling the APISIX Control API to push route weights and rate limiting thresholds to the APISIX gateway in real time, without requiring a gateway restart.
[0039] The dynamic weight calculator aggregates data to calculate the service capability score and dynamic rate limiting threshold for each service instance.
[0040] Based on the comparison results of aggregated service level performance metrics and preset thresholds, the traffic control plugin parameters on the API gateway are dynamically enabled, disabled, or adjusted.
[0041] S3: When a client request reaches the APISIX gateway, the enhanced load balancer selects a service instance based on real-time routing weights, while the traffic control plugin implements multi-dimensional traffic control based on the rate limiting threshold.
[0042] The traffic control plugin implements multi-dimensional traffic control based on the rate limiting threshold, including: the traffic control plugin implements multi-dimensional traffic control based on IP address, user identity, service type and API endpoint, and dynamically adjusts the rate limiting threshold according to the overall system load, business priority and user level.
[0043] This step involves intelligent routing and flow control. When a client request reaches the APISIX gateway, the enhanced load balancer selects the healthiest backend instance based on the cached instance weights. Simultaneously, the flow control plugin performs rate limiting based on dynamic thresholds.
[0044] Based on APISIX, as the actual implementer, a complete APISIX deployment is required to receive dynamic configurations from the capability marketplace. When a request arrives, the gateway no longer uses a fixed load balancing algorithm but instead selects routes based on real-time updated service instance weight configurations. Simultaneously, it strictly enforces dynamic rate limiting and circuit breaker rules, achieving fine-grained control over traffic.
[0045] Prior to S1, shared resource service capabilities were registered in the service registry, and service access addresses and related parameters were configured. The registration process did not require modification of the backend service capability code. Regarding service capabilities, no backend modifications were needed; only the capability addresses requiring access for monitoring and control needed to be registered, and the relevant parameters were configured during capability registration.
[0046] Service Registry and Discovery Center: Based on service registries such as Nacos, Consul, and Eureka, it maintains a healthy list of service instances.
[0047] When the error rate of a service instance consistently exceeds a preset threshold, it is automatically marked as "unhealthy" and temporarily removed from the upstream service list to implement circuit breaker protection. For service instances that recover after being circuit-broken, they are reintroduced with their initial weight for gradual traffic recovery. When the error rate of a service instance remains excessively high, it can be automatically marked as "unhealthy" and temporarily removed from the upstream (circuit breaker). After recovery, it is reintroduced with a lower initial weight for trial traffic import (graceful recovery).
[0048] After S3 is executed, S1 is executed again. The actual effect of routing and traffic control is monitored. The results of routing and rate limiting will affect the metrics of backend service instances. The results are fed back to the metric collection step to form a continuous optimization loop.
[0049] In this embodiment, during the APISIX request processing lifecycle, a dynamic decision engine is embedded through a custom plugin or a relevant routing policy is configured. This enables real-time interaction with external service registries, monitoring systems, and policy servers, thereby dynamically modifying routing and traffic control behaviors at runtime. Example
[0050] like Figure 3 and Figure 4 As shown, the APISIX-based dynamic routing and traffic control system provided in this embodiment is a system corresponding to the APISIX-based dynamic routing and traffic control method in Embodiment 1, and includes: Metrics collection module: Used to continuously collect operational metrics of each service instance through the service instance collector and store them in the time series database.
[0051] The strategy calculation and distribution module is used to periodically query the operating indicators within a specified time window from the time series database through a dynamic weight calculator, calculate the service capability score of each service instance based on the weighted formula, and determine the health status level of the corresponding service instance based on the service capability score; based on the service capability score and health status level, dynamically adjust the routing weight and rate limiting threshold of the corresponding service instance, and push it to the APISIX gateway.
[0052] Routing and flow control module: When a client request arrives at the APISIX gateway, the enhanced load balancer selects a service instance based on real-time routing weights, while the flow control plugin implements multi-dimensional flow control based on flow limiting thresholds.
[0053] This embodiment can collect service operation metrics in real time and dynamically calculate service capability scores; dynamically adjust service routing weights and rate limiting thresholds based on service capability scores; implement multi-dimensional traffic control through the APISIX plugin; and implement real-time configuration push and version management based on etcd. The custom decision plugin, as the core logic of the custom plugin, can be built into the plugin or be an independent external service.
[0054] This embodiment addresses issues such as the inability to dynamically route, monitor traffic, and control access during the sharing and invocation of resource service capabilities in resource systems, enabling unified management and sharing of resource service capabilities. It achieves intelligent selection of the optimal upstream node based on configuration rules, real-time service instance status, and external rules without requiring a restart; it implements context-aware dynamic traffic control, dynamically adjusting rate limiting and circuit breaker strategies based on overall system load, business priority, and user level; and it improves the overall elasticity, availability, and resource utilization of the system, achieving intelligent traffic scheduling.
[0055] The dynamic routing and flow control method and system according to APISIX of the present invention have been described above by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various modifications can be made to the dynamic routing and flow control method and system for APISIX proposed in the present invention without departing from the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the contents of the appended claims.
Claims
1. A dynamic routing and flow control method based on APISIX, characterized in that, Includes the following steps: S1: Continuously collect the operational metrics of each service instance through the service instance collector and store them in the time series database; S2: Periodically query the operating indicators within a specified time window from the time series database using a dynamic weight calculator, calculate the service capability score of each service instance based on the weighted formula, and determine the health status level of the corresponding service instance based on the service capability score; Based on the service capability score and health status level, the routing weight and rate limiting threshold of the corresponding service instance are dynamically adjusted and pushed to the APISIX gateway; S3: When a client request reaches the APISIX gateway, the enhanced load balancer selects a service instance based on real-time routing weights, while the traffic control plugin implements multi-dimensional traffic control based on the rate limiting threshold.
2. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, In S1, the operating metrics include at least one of the following: CPU utilization, memory utilization, request response latency, and HTTP error code ratio.
3. The dynamic routing and flow control method based on APISIX according to claim 2, characterized in that, In S2, the service capability score of each service instance is calculated based on a weighted formula, including: service capability score S=w1×f1+w2×f2+w3×f3+w4×f4+..., where f1-f4 are normalization functions that map different dimensional operating indicators to a unified score range, and w1-w4 are weight coefficients that are dynamically adjusted according to the service type.
4. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, In S2, the health status levels include four levels: healthy, sub-healthy, dangerous, and offline.
5. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, In S2, pushing to the APISIX gateway includes: pushing the route weight and rate limiting threshold to the APISIX gateway in real time via the APISIX Control API, without restarting the gateway.
6. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, In S3, the traffic control plugin implements multi-dimensional traffic control based on the rate limiting threshold, including: the traffic control plugin implements multi-dimensional traffic control based on IP address, user identity, service type and API endpoint, and dynamically adjusts the rate limiting threshold according to the overall system load, business priority and user level.
7. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, Before step S1, the resource service capabilities that need to be shared are registered in the service registry, and the service access address and related parameters are configured; the registration process does not require modification of the backend service capability code.
8. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, When the error rate of a service instance continues to exceed a preset threshold, it is automatically marked as "unhealthy" and temporarily removed from the upstream service list to achieve circuit breaker protection. For service instances that recover after being circuit broken, they are reintroduced with the initial weight for gradual traffic recovery.
9. The dynamic routing and flow control method based on APISIX according to claim 1, characterized in that, After S3 is executed, S1 is executed again.
10. A dynamic routing and traffic control system based on APISIX, characterized in that, include: Metrics collection module: Used to continuously collect operational metrics of each service instance through the service instance collector and store them in the time series database; The strategy calculation and distribution module is used to periodically query the operating indicators within a specified time window from the time series database through a dynamic weight calculator, calculate the service capability score of each service instance based on the weighted formula, and determine the health status level of the corresponding service instance based on the service capability score. Based on the service capability score and health status level, the routing weight and rate limiting threshold of the corresponding service instance are dynamically adjusted and pushed to the APISIX gateway; Routing and flow control module: When a client request arrives at the APISIX gateway, the enhanced load balancer selects a service instance based on real-time routing weights, while the flow control plugin implements multi-dimensional flow control based on flow limiting thresholds.