A load balancing service method

Abstract

The application provides a load balancing service method, and relates to the technical field of load balancing, which comprises the following steps: in each statistical period, calculating the response time weighted average of each instance based on the current response time and the historical response time of each instance, calculating the instance weight according to the response time weighted average of each instance, and obtaining the weight interval; periodically updating the instance weight and the weight interval; calculating the random weight by using the random number and the maximum weight in all instances; starting from the weight interval with the minimum value, sequentially judging the relationship between the upper limit of the weight interval of each instance and the random weight, and when the upper limit of the weight interval of the instance is greater than or equal to the random weight, selecting to send the current request to the instance. The application can rapidly reduce the weight of the problem server by improving the response time weight algorithm, dynamically adjust the weight to avoid the performance fluctuation caused by the too fast weight change, and smoothly recover the weight of the problem instance to avoid the resource waste.

Description

Technical Field

[0001] This invention relates to the field of load balancing technology, and in particular to a load balancing service method. Background Technology

[0002] Load balancing is a crucial technology in modern distributed systems. It improves system availability, reliability, and performance by distributing requests rationally across multiple servers. Common application-layer load balancing algorithms include: Round Robin, Weighted Round Robin, Random, Weighted Random, Consistent Hashing, Response Time, Least Connections, and Source IP Hashing.

[0003] In scenarios where the optimal microservice instance is selected based on its response time and weight, a mature existing technology is the Weighted Response Time Rule in Netflix's open-source load balancing tool, Ribbon. This algorithm dynamically adjusts the weights of each service instance based on its average response time; instances with shorter response times have higher weights and a greater probability of being selected. Its workflow is as follows: Figure 1 As shown, specifically:

[0004] (1) Average response time: Record the response time at the end of each request and update relevant statistics (such as number of requests, total response time). The average response time is calculated from the statistics: average response time = total response time / number of requests.

[0005] (2) Instance weights:

[0006] First, calculate the total response time (totalResponseTime) of all microservice instances.

[0007] Then, a weight is calculated for each microservice instance. The formula for calculating the weight is: Weight = totalResponseTime (average response time of the instance) + weightSoFar, where weightSoFar is the accumulated weight value.

[0008] For example, suppose there are four microservice instances A, B, C, and D, with average response times of 10ms, 50ms, 100ms, and 200ms, respectively:

[0009] Total response time = 10 + 50 + 100 + 200 = 360 ms.

[0010] The weights are calculated as follows:

[0011] Example A: 360 - 10 + 0 = 350 (weightSoFar = 0)

[0012] Example B: 360 - 50 + 350 = 660 (weightSoFar = 350)

[0013] Example C: 360 - 100 + 660 = 920 (weightSoFar = 660)

[0014] Example D: 360 - 200 + 920 = 1080 (weightSoFar = 920).

[0015] The weight range for each instance is as follows:

[0016] Example A: [0, 350]

[0017] Example B: (350, 660)

[0018] Example C: (660, 920)

[0019] Example D: (920, 1080).

[0020] The width of the weight interval reflects the probability of an instance being selected. Instances with shorter response times have wider weight intervals and a higher probability of being selected.

[0021] (3) Dynamically calculate weights:

[0022] The WeightedResponseTimeRule updates the weight of each instance periodically via a scheduled task. By default, the weight is updated every 30 seconds.

[0023] If an instance's response time is consistently above average, its weight will gradually decrease; conversely, if its response time is consistently below average, its weight will gradually increase.

[0024] (4) Select an instance:

[0025] A random weight value is generated by multiplying the random number [0.0, 1.0) by the last weight value: randomWeight = random * lastWeight

[0026] For example, assuming the final weight is 1080 and the random number is 0.3, then randomWeight = 0.3 * 1080 = 324

[0027] The system sequentially checks the weight values ​​of each instance to see if they are greater than or equal to the random weight value. If they are, the instance is selected. As follows:

[0028] Example A: 350. If 350 >= 324, then this example is selected.

[0029] Example B: 660

[0030] Example C: 920

[0031] Example D: 1080

[0032] (5) Examples:

[0033] Taking 10 servers (instances) as an example (500ms is the normal response time, 1000ms is a slightly longer response time, and 1500ms is an excessively long response time), the results after applying the WeightedResponseTimeRule native load balancing algorithm are shown in Table 1:

[0034] Table 1 shows the weight ratios and selection probabilities of each instance derived from the weighted response time algorithm.

[0035]

[0036] In the example above, the response time ratio of the first instance AJ is: 1:2:3:1:1:1:1:1:1:1

[0037] After aggregating the server response times, the weighting ratio of this algorithm is: 1:0.92:0.91:1:1:1:1:1:1:1:1

[0038] The response time ratio of the second round of instance AJ is: 1:1.5:2:1:1:1:1:1:1:1:1

[0039] After aggregating the server response times, the weighting ratio of this algorithm is: 1:0.95:0.95:1:1:1:1:1:1:1

[0040] It can be seen that when the number of servers in the cluster increases, the weights are basically the same.

[0041] Based on the examples above, it can be seen that although existing load balancing algorithms can allocate weights based on response time, they still have many shortcomings: (1) Insufficient penalty for problematic servers: When there are many servers or the random numbers are uneven, the weight difference is not obvious, resulting in servers with excessively long response times still receiving a large number of requests, affecting overall performance. (2) Lack of dynamic adjustment capability: Existing algorithms fail to fully consider the historical performance of servers and cannot dynamically adjust weights to adapt to changes in server status. (3) Insufficient recovery capability: When the response time of a problematic server returns to normal, existing algorithms cannot quickly restore its weight, resulting in a waste of server resources.

[0042] Therefore, there is an urgent need to provide a better load balancing service method to solve the problems of insufficient server penalty, poor dynamic adjustment capability, and insufficient recovery capability. Summary of the Invention

[0043] To address the aforementioned issues, this invention provides a load balancing service method. By improving the response time weighting algorithm, it can quickly reduce the weight of problematic servers. When a server's response time increases, its weight can be rapidly reduced, decreasing request allocation. It can also dynamically adjust the weight based on the server's historical performance, avoiding performance fluctuations caused by rapid weight changes. Furthermore, it can smoothly restore the weight when a problematic server returns to normal, avoiding resource waste.

[0044] To achieve the above objectives, the present invention provides a load balancing service method, comprising:

[0045] Within each statistical period, the weighted average response time of each instance is calculated based on the current response time and the historical response time.

[0046] Within each statistical period, the instance weight is calculated based on the weighted average of the response times of each instance, and the weight range is obtained.

[0047] The instance weights and weight ranges are updated periodically.

[0048] Calculate the random weight using a random number and the maximum weight across all instances;

[0049] Starting with the smallest weight range, the relationship between the upper limit of the weight range of each instance and the random weight is determined sequentially. If the upper limit of the weight range of an instance is greater than or equal to the random weight, the current request is sent to that instance.

[0050] As a further improvement of the present invention, the instances with the highest and lowest survival requests are obtained from all instances;

[0051] If the instance selected to send the current request is the one with the highest number of live requests, then the current request is sent to the instance with the lowest number of live requests.

[0052] As a further improvement of the present invention, within each statistical period, a weighted average response time of each instance is calculated based on the current response time and historical response time of each instance; including:

[0053] Set the weight ratio a:b between the current response time and the historical response time;

[0054] The formula for calculating the weighted average of the instance responses is:

[0055] T = a × Tcurrent + b × Thistory

[0056] In the formula:

[0057] Tcurrent represents the response time of the instance in the current statistical period;

[0058] Thistory represents the weighted average of the historical response times of an instance.

[0059] As a further improvement of the present invention, within each statistical period, the instance weight is calculated based on the weighted average of the response times of each instance, including:

[0060] The formula for calculating the instance weight is:

[0061] Weight = (1000 / (weighted average of instance response times + 1))^2 + weightSoFar

[0062] In the formula,

[0063] weightSoFar represents the accumulated weight value;

[0064] 1000 / (weighted average of instance response times + 1) means taking the reciprocal of the response time in seconds to avoid variable overflow.

[0065] As a further improvement of the present invention, the weight interval is obtained as follows:

[0066] For any instance, if there exists an instance among all instances whose weight value c is less than and closest to the current instance's weight value d, then the weight interval of that instance is (c, d].

[0067] If it does not exist, the weight range of the instance is (0, d).

[0068] As a further improvement of the present invention

[0069] The wider the weighted interval, the higher the probability of it being selected.

[0070] As a further improvement to the present invention, the instance weights are periodically updated; including:

[0071] After each statistical period ends, obtain the instance weight and weight range of each instance obtained in the last calculation within that statistical period, and update the instance weight and weight range of each instance.

[0072] As a further improvement to the present invention, the random weight is calculated using a random number and the maximum weight among all instances, as shown in the formula:

[0073] Random weight = random number * maximum weight.

[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0075] This invention improves the weighted average response time calculation method by combining the current response time and historical response time, smoothing out weight changes. It provides a reciprocal square weight calculation method by calculating the inverse square of the response time, which quickly reduces the weight of problematic servers, thereby improving the response time weight algorithm. Based on the improved response time weight algorithm and the dynamic adjustment and recovery mechanism of weight, the weight can be dynamically adjusted according to the historical performance of the service instance, and the weight can be smoothly restored.

[0076] The advantages of this invention also include: (1) Rapidly reducing the weight of problematic servers: When the server response time increases, the weight decreases rapidly, reducing request allocation. (2) Dynamically adjusting weights: The weights are dynamically adjusted based on the server's historical performance to avoid performance fluctuations caused by rapid weight changes. (3) Smoothly restoring weights: When the problematic server returns to normal, the weights can be smoothly restored, avoiding resource waste. (4) Improving overall cluster performance: By reasonably allocating requests, the impact of problematic servers on cluster performance is reduced, improving overall performance. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of an existing load balancing service method described in the background art of this invention;

[0078] Figure 2 This is a schematic diagram of a load balancing service method disclosed in one embodiment of the present invention. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] The present invention will now be described in further detail with reference to the accompanying drawings:

[0081] like Figure 2 As shown, the present invention provides a load balancing service method, comprising:

[0082] S1. Within each statistical period, calculate the weighted average response time of each instance based on the current response time and the historical response time of each instance.

[0083] This includes:

[0084] Set the weight ratio a:b between the current response time and the historical response time;

[0085] The formula for calculating the weighted average of instance responses is:

[0086] T = a × Tcurrent + b × Thistory

[0087] In the formula:

[0088] Tcurrent represents the response time of the instance in the current statistical period;

[0089] Thistory represents the weighted average of the historical response times of an instance.

[0090] Furthermore,

[0091] In this invention, within each statistical period, the weighted average response time of each instance is calculated based on the current response time and the historical response time of each instance. This can prevent performance fluctuations of problematic server nodes after the weight difference increases, and smooth the changes in response time.

[0092] Specifically,

[0093] Set the weight ratio of current response time to historical response time to 2:8;

[0094] The calculation formula is: T = 0.2 × Tcurrent + 0.8 × Thistory;

[0095] If an instance responds 4 times within a statistical period, the weighted average response time for each response is calculated as follows:

[0096] The response time for the first request is 10ms, so T = 0.2 * 10 + 0.8 * 0 = 2

[0097] The response time for the second request is 50ms, therefore T = 0.2 * 50 + 0.8 * 2 = 11.6

[0098] The response time for the third request is 100ms, therefore T = 0.2 * 100 + 0.8 * 11.6 = 29.28

[0099] The response time for the 4th request is 200ms, so T = 0.2 * 200 + 0.8 * 29.28 = 63.424.

[0100] S2. Within each statistical period, calculate the instance weight based on the weighted average response time of each instance, and obtain the weight range.

[0101] in,

[0102] The formula for calculating instance weights is:

[0103] Weight = (1000 / (weighted average of instance response times + 1))^2 + weightSoFar

[0104] In the formula,

[0105] weightSoFar represents the accumulated weight value;

[0106] 1000 / (weighted average of instance response times + 1) means taking the reciprocal of the response time in seconds to avoid variable overflow.

[0107] The weighted intervals are obtained, including:

[0108] For any instance, if there exists an instance among all instances whose weight value c is less than and closest to the current instance's weight value d, then the weight interval of that instance is (c, d].

[0109] If it does not exist, the weight range of the instance is (0, d).

[0110] The width of the weight interval reflects the probability of an instance being selected. Instances with shorter response times have wider weight intervals and a higher probability of being selected.

[0111] Specifically,

[0112] Suppose there are four service instances A, B, C, and D, whose weighted average response times are 2, 11.6, 29.28, and 63.424, respectively.

[0113] Instance weights are calculated as follows:

[0114] Example A: (1000 / (2+1))^2+0=111111.11(weightSoFar=0)

[0115] Example B: (1000 / (11.6+1))^2+111111.11=117409.93(weightSoFar=111111.11)

[0116] Example C: (1000 / (29.28+1))^2+117409.93=118500.58(weightSoFar=117409.93)

[0117] Example D: (1000 / (63.424+1))^2+118500.58=118741.52(weightSoFar=118500.58).

[0118] The weight range for each instance is as follows:

[0119] Example A: [0, 111111.11]

[0120] Example B: (111111.11, 117409.93)

[0121] Example C: (117409.93, 118500.58)

[0122] Example D: (118500.58, 118741.52).

[0123] S3. Periodically update instance weights and weight ranges;

[0124] This includes:

[0125] After each statistical period ends, obtain the instance weight and weight range of each instance obtained in the last calculation within that statistical period, and update the instance weight and weight range of each instance.

[0126] Furthermore,

[0127] In this application, the instance weights and weight ranges are updated periodically (i.e. dynamically). When a service instance in a problematic server recovers to normal, its historical response time data still exists. After calculating the weighted average of the instance response time according to the formula T = a × Tcurrent + b × Thistory, its weight value can be obtained at any time. After one or more statistical periods of recovery response time, the weight of the service instance can be gradually restored, avoiding large fluctuations in the weight of the service instance, which could lead to unreasonable request allocation.

[0128] S4. Obtain the instances with the highest and lowest liveness requests from all instances;

[0129] in,

[0130] The instance with the most liveness requests: The instance that receives the most liveness check requests within a certain period of time;

[0131] The instance with the fewest liveness check requests: The instance that receives the fewest liveness check requests within a certain period of time;

[0132] The liveness check is initiated by the load balancer or ingress controller. The request distribution is affected by the load balancing algorithm. If an instance is frequently in a "sub-healthy" state (such as occasional timeouts), the liveness check mechanism will increase the retry frequency (such as the exponential backoff strategy), resulting in the number of liveness requests for that instance being significantly higher than other stable instances; that is, the instance with the highest number of liveness requests may be in a sub-healthy state.

[0133] S5. Calculate the random weight using a random number and the maximum weight among all instances;

[0134] in,

[0135] The random weight is calculated using a random number and the maximum weight among all instances, using the following formula:

[0136] Random weight = random number * maximum weight.

[0137] Specifically,

[0138] A random weight value is generated by multiplying the final weight value by a random number [0.0, 1.0). Assuming the final weight value is 118741.52 and the random number is 0.3, then randomWeight = 0.3 * 118741.52 = 35622.456.

[0139] S6. Starting from the weight range with the smallest value, determine the relationship between the upper limit of the weight range of each instance and the random weight. If the upper limit of the weight range of an instance is greater than or equal to the random weight, then select to send the current request to that instance.

[0140] Specifically,

[0141] The system sequentially checks the weight values ​​of each instance to see if they are greater than or equal to the random weight value. If they are, the instance is selected. As follows:

[0142] Example A: 111111.11, 111111.11>=35622.456, then select this example.

[0143] Example B: 117409.93

[0144] Example C: 118500.58

[0145] Example D: 118741.52

[0146] S7. If the instance selected to send the current request is the instance with the highest number of live requests, then send the current request to the instance with the lowest number of live requests.

[0147] in,

[0148] Avoid sending requests to instances that are in poor condition, ensure sufficient response time for requests, and achieve better load balancing.

[0149] Example:

[0150] Taking 10 service instances as an example (500ms is the normal response time, 1000ms is a slightly longer response time, and 1500ms is an excessively long response time), the load balancing service method of this invention is applied, and the results are shown in Table 2:

[0151] Table 2 Instance Weight Values ​​and Instance Selection Results

[0152]

[0153]

[0154] As can be seen from Table 2:

[0155] (1) Servers with slow response times can have their weight for receiving requests significantly reduced by the load balancing service method of the present invention.

[0156] (2) The statistical algorithm will gradually restore the request volume based on historical performance to ensure that the problematic server recovers smoothly and avoids fluctuations.

[0157] Advantages of this invention:

[0158] This invention improves the weighted average response time calculation method by combining the current response time and historical response time, smoothing out weight changes. It provides a reciprocal square weight calculation method by calculating the inverse square of the response time, which quickly reduces the weight of problematic servers, thereby improving the response time weight algorithm. Based on the improved response time weight algorithm and the dynamic adjustment and recovery mechanism of weight, the weight can be dynamically adjusted according to the historical performance of the service instance, and the weight can be smoothly restored.

[0159] The advantages of this invention also include: (1) Rapidly reducing the weight of problematic servers: When the server response time increases, the weight decreases rapidly, reducing request allocation. (2) Dynamically adjusting weights: The weights are dynamically adjusted based on the server's historical performance to avoid performance fluctuations caused by rapid weight changes. (3) Smoothly restoring weights: When the problematic server returns to normal, the weights can be smoothly restored, avoiding resource waste. (4) Improving overall cluster performance: By reasonably allocating requests, the impact of problematic servers on cluster performance is reduced, improving overall performance.

[0160] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A load balancing service method, characterized in that, include: Within each statistical period, the weighted average response time of each instance is calculated based on the current response time and the historical response time. Specifically, this includes setting the weight ratio a:b between the current response time and the historical response time. The formula for calculating the weighted average response time of an instance is: T = a × Tcurrent + b × Thistory, where Tcurrent represents the response time of the instance in the current statistical period, and Thistory represents the weighted average response time of the instance in the historical period. Within each statistical period, the instance weight is calculated based on the weighted average response time of each instance. The instance weight calculation formula is: weight = (1000 / (weighted average response time of instances + 1))^2 + weightSoFar, where weightSoFar represents the accumulated weight value; 1000 / (weighted average response time of instances + 1) represents taking the reciprocal of the response time in seconds to avoid variable overflow; and the weight interval is obtained, specifically including: for any instance, if there is an instance among all instances whose weight value c is less than and closest to the current instance's weight value d, then the weight interval of that instance is (c, d]; if there is no such instance, then the weight interval of that instance is (0, d]. The instance weights and weight ranges are updated periodically. Calculate the random weight using a random number and the maximum weight across all instances; Starting with the smallest weight range, the relationship between the upper limit of the weight range of each instance and the random weight is determined sequentially. If the upper limit of the weight range of an instance is greater than or equal to the random weight, the current request is sent to that instance.

2. The load balancing service method according to claim 1, characterized in that: Retrieve the instances with the highest and lowest liveness requests from all instances; If the instance selected to send the current request is the one with the highest number of live requests, then the current request is sent to the instance with the lowest number of live requests.

3. The load balancing service method according to claim 1, characterized in that: The wider the weighted interval, the higher the probability of it being selected.

4. The load balancing service method according to claim 1, characterized in that: Periodically update instance weights; including: After each statistical period ends, obtain the instance weight and weight range of each instance obtained in the last calculation within that statistical period, and update the instance weight and weight range of each instance.

5. The load balancing service method according to claim 1, characterized in that: The random weight is calculated using a random number and the maximum weight among all instances, using the following formula: Random weight = random number * maximum weight.

Images