Self-adaptive dynamic flow management system, method and product

Abstract

The invention discloses a self-adaptive dynamic flow management system, method and product, relates to the field of cloud native, and is used for improving the system stability and reducing the request failure rate. According to the invention, the operation state data of the micro-service instance is monitored in real time; dynamically calculating an optimal flow distribution mode according to the running state data monitored in real time; updating a load balancing strategy and a traffic weight according to the optimal traffic distribution mode; and detecting the health state of the micro-service instance according to the running state data monitored in real time, triggering fusing or rejecting flow distribution for the corresponding micro-service instance according to a detection result, and starting a fusing recovery mechanism after the fusing is triggered. Through refined dynamic traffic scheduling, the throughput of the system is improved, the request delay is greatly reduced, and the overall performance and response speed of the system are effectively improved. In addition, flow distribution is dynamically adjusted according to the real-time state of the instance, and the problems of resource waste and load imbalance are avoided.

Description

Technical Field

[0001] The present invention relates to the field of cloud native technology, and in particular to an adaptive dynamic traffic management system, method, and computer program product for a service grid system. Background Art

[0002] Amid the rapid development of cloud-native architectures, microservices have become the mainstream approach for building complex applications. Microservices architectures split traditional monolithic applications into multiple independent services, each responsible for specific functionality and communicating over a network. With the widespread adoption of microservices architectures, inter-microservice communication and traffic management have become key challenges in cloud computing environments. In high-concurrency scenarios, traffic scheduling must not only ensure service load balancing but also be adaptive when microservice instances fail or experience performance fluctuations, ensuring that traffic can be quickly and appropriately switched between healthy instances. Existing traffic scheduling strategies often use static weights or simple round-robin mechanisms, making them difficult to cope with dynamically changing traffic demands. Furthermore, issues such as traffic skew, uneven resource utilization, and service avalanches place higher demands on traffic scheduling.

[0003] Istio, a mainstream service mesh solution, provides traffic control, observability, and security management for microservices in Kubernetes clusters. Load balancing is a core feature of Istio. The goal of load balancing is to distribute traffic across multiple service instances to optimize system performance, improve resource utilization, and maintain service stability and reliability in high-concurrency scenarios. However, Istio load balancing policies are unaware of the operating status of microservice instances, making dynamic load balancing impossible.

[0004] Other known existing technologies also have some shortcomings in traffic management: The Chinese patent document with publication number CN119544617A proposes a method and device for traffic management, which first obtains the operating parameters of each microservice node. Secondly, in response to the obtained operating parameters of any target microservice node, it is determined based on the operating parameters whether the target microservice node meets the flow limiting conditions corresponding to the operating parameters. Then, if so, the resource contribution corresponding to multiple functions is calculated, and the resource contribution is used to represent the contribution of the resources consumed by the function to the service provided by the target microservice node. Finally, based on the resource contribution, the target function to be flow limited is determined from multiple functions, and the target function is flow limited. However, since this method relies too much on static operating parameters and lacks automatic recovery mechanism and dynamic adjustment capability, it may perform poorly under high load and burst traffic conditions.

[0005] A Chinese patent publication with publication number CN119520402A proposes a method and system for adaptive traffic scheduling based on business characteristics in a multi-cloud environment. The method includes: randomly sending user traffic requests to any cloud service in the multi-cloud environment; using a deep learning model to obtain monitoring data from each cloud service in the multi-cloud environment, and based on the user traffic request and the set traffic balancing rules, determining whether the current user traffic request needs to be forwarded; if the judgment result is yes, the current user traffic request is forwarded to the target cloud service; if the judgment result is no, the current user traffic request is received and processed by the current cloud service. However, this method relies on a deep learning model for traffic scheduling, which has a high computational cost and may suffer from cold start problems, resulting in unstable scheduling results in the initial stage. In addition, its traffic balancing rules are relatively fixed and lack the ability to adapt to instance performance fluctuations in real time, and may not be able to adjust quickly when the load changes.

[0006] A Chinese patent publication, CN119276873A, proposes a load access control method based on a service grid. This method provides microservices through a service grid, simplifying their development, deployment, management, and maintenance. The service grid provides functions such as traffic management, service discovery, network policy, authentication and authorization, and monitoring, ensuring that communication between microservices is more reliable, efficient, and secure. Load control enables precise management and control of microservice traffic, intelligently routing requests to the most appropriate microservice instance based on the request characteristics and the microservice status. However, this method primarily relies on a static load control strategy, and load balancing may experience certain delays. Summary of the Invention

[0007] The purpose of the present invention is to provide an adaptive dynamic traffic management system, method and product to address all or part of the above-mentioned problems, thereby improving system stability and reducing request failure rate through dynamic load balancing and traffic scheduling.

[0008] The technical solution adopted in the present invention is as follows: An adaptive dynamic traffic management system, comprising: The data monitoring module monitors the running status data of the microservice instance in real time, including request rate, response time and error rate; The load balancing decision module dynamically calculates the optimal traffic distribution method based on real-time monitored operating status data, including dynamic selection of load balancing strategies and dynamic allocation of traffic weights; A traffic scheduling execution module updates the load balancing strategy and traffic weight according to the optimal traffic distribution method; The circuit breaker and health detection module detects the health status of microservice instances based on real-time monitored operating status data, triggers circuit breaking or eliminates traffic distribution for the corresponding microservice instances based on the detection results, and starts the circuit breaker recovery mechanism after the circuit breaker is triggered.

[0009] Furthermore, the data monitoring module monitors the running status data of the microservice instance in real time according to the following configuration: Trigger Istio to intercept communication data between all microservice instances through the Envoy proxy and send the indicator data to Prometheus; Call the Prometheus API, query indicator data through Prometheus, and calculate the operating status data of the microservice instance.

[0010] Furthermore, the load balancing decision module dynamically selects a load balancing strategy based on the following configurations: Calculate the response time variance / standard deviation of each microservice instance based on the running status data; If the response time variance / standard deviation is lower than the first threshold, the load balancing strategy uses ROUND_ROBIN; If the response time variance / standard deviation is higher than the second threshold, the load balancing strategy uses LEAST_CONN; If the response time variance / standard deviation is between the first threshold and the second threshold, the load balancing strategy uses RANDOM.

[0011] Furthermore, the load balancing decision module dynamically allocates traffic weights according to the following configuration: Calculate the inverse of the response time of each microservice instance based on the running status data; The inverse of the response time is normalized to obtain the traffic weight.

[0012] Furthermore, the traffic scheduling execution module updates the load balancing strategy according to the following configuration: Call the Kubemetes API, and automatically update the Destination Rule through Kubemetes according to the dynamically calculated optimal traffic distribution method. The Destination Rule dynamically adjusts the load balancing strategy of the microservice instance according to the load balancing strategy dynamically selected in the optimal traffic distribution method.

[0013] Furthermore, the traffic scheduling execution module updates the traffic weight according to the following configuration: Call the Kubemetes API, and Kubemetes automatically updates the Virtual Service based on the dynamically calculated optimal traffic distribution method. The Virtual Service dynamically adjusts the traffic weight of the microservice instance based on the traffic weight dynamically allocated in the optimal traffic distribution method.

[0014] Furthermore, the circuit breaker and health detection module triggers circuit breaking for the microservice instance and initiates the circuit breaker recovery mechanism after the circuit breaker is triggered according to the following configuration: The error rate of the microservice instance is detected based on the real-time monitored operating status data. If the error rate is higher than the third threshold, the circuit breaker is triggered, and the traffic allocation to the microservice instance is suspended for a first period of time. At the same time, the circuit breaker recovery mechanism is started. After the first period of time is reached, the error rate of the microservice instance is re-evaluated. If the error rate is lower than the third threshold, the microservice instance is restored to participate in traffic distribution again, otherwise the circuit breaker is triggered again.

[0015] Furthermore, the circuit breaker and health detection module removes traffic allocation from the microservice instance according to the following configuration: Call the Kubernetes API to check the Pod status of the microservice instance through Kubernetes. If the check result shows that the microservice instance is abnormal, the microservice instance is removed from participating in traffic distribution.

[0016] On the other hand, the present invention also proposes an adaptive dynamic traffic management method, which includes: Real-time monitoring of the operating status data of microservice instances; Based on real-time monitored operating status data, the optimal traffic distribution method is dynamically calculated, including the dynamic selection of load balancing strategies and the dynamic allocation of traffic weights; Update the load balancing strategy and traffic weight according to the optimal traffic distribution method; The health status of the microservice instance is detected based on the real-time monitored operating status data, and the circuit breaker or traffic distribution is triggered for the corresponding microservice instance based on the detection results, and the circuit breaker recovery mechanism is started after the circuit breaker is triggered.

[0017] In another aspect, the present invention further provides a computer program product, comprising a computer program, which can execute the above-mentioned adaptive dynamic traffic management method when executed by a processor.

[0018] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: The present invention can effectively improve the system throughput, reduce the request pressure of high-load microservice instances through dynamic regulation based on real-time data, reduce timeout problems caused by resource bottlenecks, avoid resource waste and load imbalance problems, and improve system stability; at the same time, combined with the fuse mechanism and health check, it reduces the impact of abnormal microservice instances on the system and improves the request success rate; the design of the fuse recovery mechanism significantly improves the fault tolerance and automatic recovery capabilities of the system, avoiding traffic interruptions and service unavailability caused by service anomalies. In addition, the intelligent load balancing strategy of the present invention can adaptively adjust according to different traffic patterns, improve scheduling efficiency, reduce manual intervention costs, and is suitable for traffic management of large-scale microservice clusters. It can be widely used in high-concurrency business scenarios such as cloud computing, finance, and the Internet. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will now be described by way of example with reference to the accompanying drawings, in which: Figure 1 It is a structural diagram of the adaptive dynamic traffic management system.

[0020] Figure 2 It is the main flow chart of the adaptive dynamic traffic management method.

[0021] Figure 3 This is a flowchart of the operation of the adaptive dynamic traffic management method in a specific embodiment.

[0022] Figure 4 This is a comparison chart of the average response time of this application's adaptive dynamic traffic management solution and native Istio.

[0023] Figure 5 This is the concurrency-delay relationship diagram in the adaptive dynamic traffic management solution of this application.

[0024] Figure 6 This is the concurrency-weight relationship diagram in the adaptive dynamic traffic management solution of this application.

[0025] Figure 7 This is the relationship diagram between the number of concurrent connections and the proportion of faulty traffic in the adaptive dynamic traffic management solution of this application.

[0026] Figure 8 This is the fault injection experiment result diagram. DETAILED DESCRIPTION

[0027] All features disclosed in this specification, or all steps in the disclosed methods or processes, except mutually exclusive features and / or steps, can be combined in any manner.

[0028] Any feature disclosed in this specification (including any appended claims and abstract), unless otherwise stated, may be replaced by other equivalent or similar features. In other words, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0029] With the rapid development of cloud-native applications and microservice architectures, traditional load balancing and traffic scheduling solutions often have difficulty adapting to the needs of high-concurrency, heterogeneous environments, resulting in poor system performance in high-load scenarios, and even service interruptions and performance bottlenecks. This application embodiment proposes an adaptive dynamic traffic management system, method, and product to address problems such as uneven traffic, load fluctuations, and fault handling in current microservice architectures: 1) Inaccurate traffic scheduling under high concurrency: Traditional load balancing algorithms (such as ROUND_ROBIN and LEAST_CONN) often distribute traffic based on static rules or fixed weights. They cannot adapt to load fluctuations of microservice instances and may cause some microservice instances to be overloaded while others are idle.

[0030] 2) Untimely service failure recovery: When a microservice instance fails, traditional solutions require manual intervention or long delays, and are unable to quickly isolate the failed instance and restore the service, affecting the availability and response time of the overall system.

[0031] 3) Insufficient automation and real-time performance: In a cloud-native environment, the lifecycle of microservice instances is short, and the status and load of microservice instances change at any time. Traditional traffic management solutions usually rely on manual configuration and static rules, lack real-time response mechanisms, and cannot effectively adjust traffic distribution and load balancing strategies in rapidly changing scenarios.

[0032] See attached Figure 1 The adaptive dynamic traffic management system proposed in the embodiment of the present application includes a data monitoring module, a load balancing decision module, a traffic scheduling execution module and a fuse and health detection module.

[0033] The data monitoring module is used to monitor the running status data of the microservice instance in real time. In some specific embodiments, the running status data of the microservice instance includes a data monitoring module, a load balancing decision module, a traffic scheduling execution module, and a circuit breaker and health detection module.

[0034] The data monitoring module is used to monitor the operating status data of microservice instances in real time.

[0035] The so-called running status data of the microservice instance, in some specific embodiments, includes: 1) Request rate QPS (Queries Per Second), which reflects the current traffic size and avoids load imbalance; 2) Response time RT (Response Time), which is used to evaluate the processing capacity of each microservice instance and guide traffic distribution; 3) Error rate ER (Error Rate), which refers to the 5xx failure rate, is used for anomaly detection and supports the operation of the circuit breaker mechanism.

[0036] Traditional Istio load balancing strategies cannot perceive the running status data of microservice instances. In some optional implementations of this application, the data monitoring module is configured to monitor the running status data of microservice instances in real time in the following manner: Trigger Istio to intercept communication data between all microservice instances through the Envoy proxy and send metric data to Prometheus. Metric data refers to statistical data used to calculate metrics such as latency, traffic, errors, and saturation from the intercepted communication data (i.e., traffic). For example, metric data includes the number of requests and request duration.

[0037] Call the Prometheus API to query metrics data through Prometheus and calculate the operating status data of the microservice instance. Prometheus queries the metrics data transmitted by Istio and calculates / processes the metrics data to obtain the above operating status data.

[0038] For example, the response time can be calculated using the following formula (1.1): Formula (1.1): ; Where, Represents the sum of the duration of all istio requests since the microservice instance was last started. Indicates the total number of all istio requests since the microservice instance was last started; Indicates the response time of the i-th microservice instance.

[0039] As for the request rate and error rate, they can also be obtained by querying the indicator data.

[0040] The load balancing decision module is used to dynamically calculate the optimal traffic distribution method based on real-time monitored operating status data. The process of dynamically calculating the optimal traffic distribution method includes the dynamic selection of load balancing strategies and the dynamic allocation of traffic weights.

[0041] In current traffic management methods, load balancing strategies are usually static, for example, set as ROUND_ROBIN or LEAST_CONN. In some optional embodiments of the present application, the load balancing strategy is dynamically updated among ROUND_ROBIN, LEAST_CONN, and RANDOM based on real-time monitored operating status data. Specifically, as an optional embodiment, the ROUND_ROBIN, LEAST_CONN, and RANDOM strategies are automatically switched based on the response time variance / standard deviation of the microservice instance. Specifically, the load balancing decision module dynamically selects the load balancing strategy based on the following configuration: Calculate the response time variance / standard deviation of each microservice instance based on the running status data; If the response time variance / standard deviation is lower than the first threshold (for example, when using standard deviation, the first threshold is 10 or other thresholds), it indicates that the performance of each microservice instance is similar. In this case, the load balancing strategy uses ROUND_ROBIN. If the response time variance / standard deviation is higher than the second threshold (such as a standard deviation threshold of 20 or other values), it indicates that the performance of each microservice instance fluctuates greatly, and the response time of some microservice instances is much higher than that of other microservice instances. In this case, the load balancing strategy uses LEAST_CONN. If the response time variance / standard deviation is between the first threshold and the second threshold, the load balancing strategy uses RANDOM.

[0042] As for the traffic weight distribution strategy, traffic is preferentially distributed to microservice instances with faster response times. Specifically, as an optional implementation, the load balancing decision module dynamically distributes traffic weights based on the following configuration: Calculate the inverse response time of each microservice instance based on the running status data; The inverse of the response time is normalized to obtain the traffic weight.

[0043] The weight calculation method can be obtained according to formula (1.2): Formula (1.2): ; Where n represents the total number of microservice instances, Denotes the traffic weight assigned to the i-th microservice instance. According to formula (1.2), the distribution of traffic weight is completely determined by the response time of the microservice instance, achieving a highly adaptive effect.

[0044] In some other optional implementations, the load balancing decision module may also dynamically allocate traffic weights according to the following configurations: Calculate the mean response time of all microservice instances; Traverse each microservice instance. If the response time of the microservice instance is higher than the average, reduce the traffic weight of the microservice instance by a preset step size (such as 10%). If the response time of the microservice instance is lower than the average, increase the traffic weight of the microservice instance by a preset step size.

[0045] The traffic scheduling execution module is used to update the load balancing strategy and traffic weight according to the optimal traffic distribution method (selected by the load balancing decision module).

[0046] As an optional implementation, the traffic scheduling execution module automatically updates the Virtual Service and Destination Rule through the Kubernetes API to automatically update the traffic weight and load balancing strategy based on the optimal traffic distribution method.

[0047] Specifically, the traffic scheduling execution module updates the load balancing policy according to the following configuration: Call the Kubemetes API and use Kubemetes to automatically update the Destination Rule based on the dynamically calculated optimal traffic distribution method. The Destination Rule dynamically adjusts the load balancing policy of the microservice instance (such as dynamically adjusting ROUND_ROBIN or LEAST_CONN) based on the load balancing policy dynamically selected in the optimal traffic distribution method.

[0048] The traffic scheduling execution module updates the traffic weight according to the following configuration: Call the Kubemetes API, and Kubemetes automatically updates the Virtual Service based on the dynamically calculated optimal traffic distribution method. The Virtual Service dynamically adjusts the traffic weight of the microservice instance based on the traffic weight dynamically allocated in the optimal traffic distribution method (such as dynamically adjusting the weight of the microservice instance).

[0049] The circuit breaker and health monitoring module monitors the health of microservice instances based on real-time operational data. Based on the results, it triggers circuit breaking or traffic removal for the corresponding microservice instances. After a circuit breaker is triggered, the circuit breaker recovery mechanism is activated. The circuit breaker recovery mechanism is activated only after a circuit breaker is triggered. If a microservice instance is removed from traffic distribution, the circuit breaker recovery mechanism will not be activated.

[0050] As an optional implementation, the circuit breaker and health check module triggers circuit breaking for microservice instances and initiates circuit breaker recovery mechanisms after circuit breaking according to the following configuration: The error rate of the microservice instance is detected based on the real-time monitored operating status data. If the error rate is higher than the third threshold (such as 5% or other thresholds), the circuit breaker is triggered, and the traffic distribution to the microservice instance is suspended for a first period of time (such as 30 seconds or other period of time). At the same time, the circuit breaker recovery mechanism is started. After the first period of time is reached, the error rate of the microservice instance is re-evaluated. If the error rate is lower than the third threshold, the microservice instance is restored to participate in traffic distribution again. Otherwise, the circuit breaker is triggered again.

[0051] In addition, regarding the design of traffic allocation exclusion, in some optional implementations, the circuit breaker and health detection module is configured to exclude traffic allocation for microservice instances according to the following design: Call the Kubernetes API to check the Pod status of the microservice instance through Kubernetes. If the check result shows that the microservice instance is abnormal (for example, the status is not Running), the microservice instance is removed from participating in traffic distribution.

[0052] As a preferred implementation method, the circuit breaker and health detection module is started / run before the traffic scheduling execution module, so that the update of the load balancing strategy and traffic weight of some microservice instances can be eliminated in advance, avoiding fault conflicts and further improving the stability of the system.

[0053] The above four modules run in a cycle to enable the system to reach a stable working state.

[0054] From the above design content, it can be seen that the embodiment of the present application has the following features: 1) Decision-making mechanism based on real-time data: Traditional Istio load balancing strategies are unable to perceive the operating status of service instances. However, this application uses Prometheus to collect QPS, RT, and ER, achieving accurate perception of the operating status of microservice instances. For example, if the response time of microservice instance v2 is significantly higher than that of microservice instance v1, the traffic weight of v2 is automatically reduced, reducing the request pressure on the highly loaded instance. By integrating real-time monitoring, dynamic traffic scheduling, intelligent load balancing, and fault recovery mechanisms, an automated closed-loop traffic management system is formed, which overcomes the drawbacks of traditional static traffic distribution and manual intervention, and improves the system's adaptability and robustness in high-load and abnormal scenarios.

[0055] 2) Intelligent load balancing strategy selection: Calculate the variance / standard deviation of the response time of each microservice instance and select an appropriate load balancing strategy based on the fluctuation. Select ROUND_ROBIN when the variance / standard deviation is small (indicating similar performance of each microservice instance); select LEAST_CONN when the variance / standard deviation is large (indicating that some instances have much higher response times than others); and select RANDOM when the variance / standard deviation is moderate to maintain a moderate balance.

[0056] 3) Adaptive adjustment of traffic weights: Traditional Istio load balancing mostly uses fixed weights and cannot adapt to traffic changes. This application adopts a weight distribution method based on response time to ensure that microservice instances with low response times are assigned higher weights, thereby improving system throughput. Microservice instances with high latency automatically reduce traffic to avoid overall performance degradation due to high load.

[0057] Dynamic traffic scheduling and intelligent load balancing strategies based on real-time monitoring data form an automated traffic management mechanism, optimizing system performance in various scenarios, including high and fluctuating loads, and improving service stability, performance, and resource efficiency. By dynamically calculating the traffic weights of service instances, low-latency instances receive higher weights. Based on the variance / standard deviation of instance response times, the system automatically switches between different load balancing strategies to adapt to different load scenarios, reduce the pressure on highly loaded instances, and improve overall service performance.

[0058] 4) Automatic circuit breaking and recovery for abnormal instances: If the 5xx failure rate of an instance is too high (above the third threshold), a circuit breaker is triggered, pausing traffic distribution for a certain period of time to reduce the scope of the failure. After the pause, the health status is rechecked. If it returns to normal, the circuit breaker is lifted and traffic is redistributed. Automated fault detection, circuit breaking mechanisms, and automatic recovery reduce manual intervention and improve the system's robustness to failures. Leveraging the Kubernetes API for dynamic configuration updates ensures rapid system recovery and immediate effectiveness, improving system reliability and fault tolerance.

[0059] According to the concept of this application, the adaptive dynamic traffic management method provided in the embodiment of this application includes the following process: S1. Real-time monitoring of the operating status data of microservice instances.

[0060] S2. Based on the real-time monitored operating status data, dynamically calculate the optimal traffic distribution method, including the dynamic selection of load balancing strategies and the dynamic allocation of traffic weights.

[0061] S3. Update the load balancing strategy and traffic weight according to the optimal traffic distribution method.

[0062] S4. Detect the health status of the microservice instance based on the real-time monitored operating status data, trigger a circuit breaker or remove traffic distribution for the corresponding microservice instance based on the detection results, and start the circuit breaker recovery mechanism after the circuit breaker is triggered.

[0063] It should be noted that the above steps, especially step S3 and step S4, are not performed in the order of serial numbers. Without departing from the design concept of the present invention, they can be performed in a disorderly order. For example, step S4 is performed before step S3. Figure 2 shown.

[0064] The above process is executed cyclically to make the system work in a stable state.

[0065] In some optional implementations of the above-mentioned steps S1 to S4, the features configured in different embodiments of the data monitoring module, load balancing decision module, traffic scheduling execution module and fuse and health detection module can be further optimized by referring to the previous system embodiment part, and they will not be repeated here.

[0066] In some specific embodiments, Figure 3 As shown, the adaptive dynamic traffic management method can be implemented according to the following design.

[0067]

[0068] In the above example, the "adjust traffic weight" process can also be updated according to the normalized inverse of the response time.

[0069] In addition, an embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the adaptive dynamic traffic management method of the above embodiment can be executed.

[0070] In order to verify the effectiveness of the present invention, the proposed solution was also experimentally verified in the examples of this application.

[0071] The experimental environment includes preliminary preparations such as cluster construction, service deployment, monitoring configuration, and fault injection.

[0072] The experimental environment is set up as shown in Table 1.

[0073] Table 1 Experimental environment configuration table

[0074] (1) Request response time comparison (Comparison example: Productpage service) Table 2 shows a comparison of the service request response time (in seconds) between native Istio and this solution (MDTM) at the same concurrency level.

[0075] Table 2 Request response time comparison

[0076] Table 2 The response time comparison chart corresponding to the experimental results is as follows Figure 4 shown.

[0077] According to Table 2 / Figure 4As can be seen, the proposed solution significantly reduced the average response time of the productpage service at all test concurrency levels. In a high-concurrency scenario (500 QPS), the proposed solution reduced request time by 1.04 seconds, significantly improving system throughput.

[0078] (2) Load balancing strategy optimization While native Istio uses a static load balancing strategy, this solution combines real-time monitoring data with adaptive policy adjustments to optimize the load balancing approach for each microservice. The load balancing strategy designs for both are shown in Table 3.

[0079] Table 3 Comparison of load balancing strategy experiments

[0080] This application solution dynamically adjusts the load balancing strategy, enabling the high-load details-v1 to use LEAST_CONN and productpage-v1 to use ROUND_ROBIN, optimizing traffic distribution. Dynamic strategy selection helps maximize system throughput and reduce overload on individual instances in different business scenarios.

[0081] (3) Load distribution and fault handling analysis: fault injection method This application implemented dynamic traffic scheduling across the three versions of the reviews service (v1, v2, and v3) based on response time, error rate, and fault recovery. Versions v2 and v3 experienced high latency and fault circuit breaking under high concurrency conditions. To simulate this situation, this experiment used Istio fault injection and Kubernetes resource restrictions to artificially create faults and test the application's adaptive adjustment capabilities. To test the application's dynamic load balancing effectiveness at different concurrency levels (10, 50, 100, 200, and 500), this experiment used the following three methods to inject faults into reviews v2 and v3.

[0082] A. Fault Injection-Delay: This simulates the degradation of the processing capacity of v2 and v3 under high concurrency conditions, resulting in a significant increase in response time. This is achieved by introducing a fixed delay in the Istio Virtual Service rules for v2 and v3. The implementation is as follows:

[0083] Under low concurrency (10QPS), v2 has a high latency (100ms), and v3 is basically unavailable. Under high concurrency (200+QPS), the response time of v2 and v3 increases (500ms→2000ms), and the load balancing automatically reduces the weight of v2 and v3. At 500 concurrency, v2 and v3 time out and all traffic is switched to v1. The experimental results are as follows: Figure 5 、 Figure 6 shown.

[0084] B. HTTP 5xx Fault Injection-Abort: This simulates a sharp increase in the request failure rate (5xx) in versions 2 and 3 due to insufficient server resources or application crashes. The implementation is as follows:

[0085] The results show that when the concurrency is 50, the error rate of v2 rises to 50%, and the error rate of v3 is as high as 80%, and the traffic begins to tend to v1; when the concurrency is 100, the failure rate of v2 continues to rise. This application scheme reduces the weight of v2 to 0.02, and v3 still bears 20% of the traffic; when the concurrency is 200 or above, v2 and v3 trigger the circuit breaker, and all traffic switches to v1 to ensure availability. The experimental results are as follows Figure 7 Experimental results show that when this application detects high latency, increased error rates, or instance crashes, it can dynamically adjust the load balancing strategy to ensure reasonable traffic distribution and reduce request failure rates.

[0086] (4) Load distribution and fault handling analysis: How does the MDTM solution handle faults? Automatically modify the Istio Virtual Service and Destination Rule through the Kubernetes API to make the policy effective.

[0087] The pseudo code for dynamic update of Virtual Service is as follows:

[0088] The pseudo code for dynamic update of Destination Rule is as follows:

[0089] (5) Load distribution and fault handling analysis: Fusing and recovery: The processing flow includes: traffic monitoring, intelligent decision-making, and circuit breaker triggering. Circuit breaker and recovery: If the error rate of v2 and v3 remains above 5%, a circuit breaker is triggered and traffic distribution is suspended for 30 seconds. After 30 seconds, if v2 and v3 return to normal (error rate below 3%), traffic is gradually restored. The inverse weight of each instance's response time is calculated as follows: 1. if error_rate_v2<3 and time.time() - circuit_breaker_status["v2"]>30: 2. circuit_breaker_status["v2"] = 0 The injection experiment parameters are shown in Table 4.

[0090] Table 4 Fault injection experiment parameters

[0091] The fault injection experiment results are as follows Figure 8 shown.

[0092] The above experimental results show that this application solution has the following significant advantages over the native Istio load balancing strategy: 1) Reduced response time: This application solution reduces request latency by 19% to 23%, improving user experience; 2) Optimized traffic distribution: This application solution intelligently adjusts the load balancing strategy based on service status, reducing pressure on high-load instances and fully utilizing resources on low-load instances; 3) Intelligent circuit breaking and recovery: When the 5xx error rate is too high, this application solution can automatically circuit break the high-failure instance and restore traffic to healthy instances after 30 seconds; 4) Improved success rate and throughput: Compared with native Istio, this application solution has a higher request success rate. These results demonstrate the effectiveness of this application solution in improving system stability and reducing request failure rates.

[0093] The present invention is not limited to the aforementioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims

1. An adaptive dynamic traffic management system, characterized in that: include: The data monitoring module monitors the running status data of the microservice instance in real time, including request rate, response time and error rate; The load balancing decision module dynamically calculates the optimal traffic distribution method based on real-time monitored operating status data, including dynamic selection of load balancing strategies and dynamic allocation of traffic weights; A traffic scheduling execution module updates the load balancing strategy and traffic weight according to the optimal traffic distribution method; The circuit breaker and health detection module detects the health status of microservice instances based on real-time monitored operating status data, triggers circuit breaking or eliminates traffic distribution for the corresponding microservice instances based on the detection results, and starts the circuit breaker recovery mechanism after the circuit breaker is triggered.

2. The adaptive dynamic traffic management system according to claim 1, characterized in that: The data monitoring module monitors the running status data of the microservice instance in real time according to the following configuration: Trigger Istio to intercept communication data between all microservice instances through the Envoy proxy and send the indicator data to Prometheus; Call the Prometheus API, query indicator data through Prometheus, and calculate the operating status data of the microservice instance.

3. The adaptive dynamic traffic management system according to claim 1, wherein: The load balancing decision module dynamically selects a load balancing strategy based on the following configuration: Calculate the response time variance / standard deviation of each microservice instance based on the running status data; If the response time variance / standard deviation is lower than the first threshold, the load balancing strategy uses ROUND_ROBIN; If the response time variance / standard deviation is higher than the second threshold, the load balancing strategy uses LEAST_CONN; If the response time variance / standard deviation is between the first threshold and the second threshold, the load balancing strategy uses RANDOM.

4. The adaptive dynamic traffic management system according to claim 1, wherein: The load balancing decision module dynamically allocates traffic weights according to the following configuration: Calculate the inverse of the response time of each microservice instance based on the running status data; The inverse of the response time is normalized to obtain the traffic weight.

5. The adaptive dynamic traffic management system according to claim 1 or 3, characterized in that: The traffic scheduling execution module updates the load balancing strategy according to the following configuration: Call the Kubemetes API, and automatically update the Destination Rule through Kubemetes according to the dynamically calculated optimal traffic distribution method. The Destination Rule dynamically adjusts the load balancing strategy of the microservice instance according to the load balancing strategy dynamically selected in the optimal traffic distribution method.

6. The adaptive dynamic traffic management system according to claim 1 or 3, characterized in that: The traffic scheduling execution module updates the traffic weight according to the following configuration: Call the Kubemetes API, and Kubemetes automatically updates the Virtual Service based on the dynamically calculated optimal traffic distribution method. The Virtual Service dynamically adjusts the traffic weight of the microservice instance based on the traffic weight dynamically allocated in the optimal traffic distribution method.

7. The adaptive dynamic traffic management system according to claim 1, wherein: The circuit breaker and health detection module triggers circuit breaking on the microservice instance and starts the circuit breaker recovery mechanism after the circuit breaker is triggered according to the following configuration: The error rate of the microservice instance is detected based on the real-time monitored operating status data. If the error rate is higher than the third threshold, the circuit breaker is triggered, and the traffic allocation to the microservice instance is suspended for a first period of time. At the same time, the circuit breaker recovery mechanism is started. After the first period of time is reached, the error rate of the microservice instance is re-evaluated. If the error rate is lower than the third threshold, the microservice instance is restored to participate in traffic distribution again, otherwise the circuit breaker is triggered again.

8. The adaptive dynamic traffic management system according to claim 1, wherein: The circuit breaker and health detection module removes traffic distribution from microservice instances based on the following configuration: Call the Kubernetes API to check the Pod status of the microservice instance through Kubernetes. If the check result shows that the microservice instance is abnormal, the microservice instance is removed from participating in traffic distribution.

9. An adaptive dynamic traffic management method, characterized in that: include: Real-time monitoring of the operating status data of microservice instances; Based on real-time monitored operating status data, the optimal traffic distribution method is dynamically calculated, including the dynamic selection of load balancing strategies and the dynamic allocation of traffic weights; Update the load balancing strategy and traffic weight according to the optimal traffic distribution method; The health status of the microservice instance is detected based on the real-time monitored operating status data, and the circuit breaker or traffic distribution is triggered for the corresponding microservice instance based on the detection results, and the circuit breaker recovery mechanism is started after the circuit breaker is triggered.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the adaptive dynamic traffic management method according to claim 9 can be executed.

Images