A load balancing method and apparatus

Abstract

The application discloses a load balancing method and device, and relates to the technical field of cloud computing. The method comprises the following steps: acquiring parameter information of a service end; determining real-time weights of each service end, and pre-grouping the service ends according to the real-time weights; when a calling request is received, determining the number of available addresses in each weight group by using a dynamic routing result and a pre-grouping result; generating a weight calculation result by using the real-time weights of the weight groups and the number of available addresses corresponding to the real-time weights; and determining a target calling address of the calling request according to the weight calculation result. In the application, the pre-grouping method is adopted, and a large amount of repeated calculation work in the weight calculation process is moved to background asynchronous calculation, so that the workload of traversing addresses to obtain parameters during runtime is removed, the application can be run with extremely low performance loss even when the cluster scale is particularly large, the weight calculation result is generated by combining the dynamic routing result and the pre-grouping result, and strong support is provided for preheat calculation in a large-scale deployment scenario.

Description

Technical Field

[0001] This application relates to the field of cloud computing, and more specifically, to a load balancing method and apparatus. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] Generally, for remote call frameworks in a microservice architecture, the server is often deployed on more than one machine. Therefore, when initiating a remote call, it is essential to select an address from multiple addresses for the call. The address selection process can be mainly divided into the following steps: Figure 2 The diagram illustrates two phases: route filtering and load balancing. The purpose of the route filtering phase is primarily to filter all available target addresses. For example, in an internationalized scenario, compliance requirements may necessitate calling servers within the same region. The purpose of the load balancing phase is to select an available address from multiple available target addresses for this call. Common load balancing algorithms include: random algorithms, weighted random algorithms, and consensus algorithms. Among these, weighted random algorithms can select the server address through weight calculation.

[0004] For most applications, a period of time after application startup can cause lag due to connection pool initialization, Java C1 and C2 compilation, etc. Application warm-up aims to reduce the impact of such lag on the user experience. Application warm-up works based on weight calculation in load balancing rules. During a period after application startup, its weight is reduced, allowing smaller traffic flows to the machine first to warm up the environment. However, in actual deployment scenarios, since weight calculation is performed with each request, its time complexity is O(N). This can lead to severe performance degradation when the server deployment is large-scale. Summary of the Invention

[0005] This application provides a load balancing method and apparatus to at least solve the problem in the prior art that, in a preheating scenario, when both the server's scale and the number of requests are large, the operation consumes a large amount of computing resources.

[0006] According to an aspect of the present application, a load balancing method is also provided, comprising: obtaining parameter information of service ends; the parameter information comprising addresses of the service ends, weight threshold values and preheating time lengths; determining real-time weights of the service ends based on the weight threshold values, the preheating time lengths and preheated time lengths, and pre-grouping the service ends according to the real-time weights to obtain pre-grouping results; the pre-grouping results comprising a plurality of weight groups; each weight group comprising addresses of one or more service ends with the same real-time weight; when a calling request is received, determining the number of available addresses in each weight group by using a dynamic routing result and the pre-grouping results; the dynamic routing result comprising one or more available addresses; generating a weight calculation result by using the real-time weights of the weight groups and the number of available addresses corresponding to the real-time weights; and determining a target calling address of the calling request according to the weight calculation result.

[0007] According to another aspect of the present application, a computer device is also provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method steps described above when executing the computer program.

[0008] According to another aspect of the present application, a computer readable storage medium is also provided, which stores a computer program, wherein the computer program is executable on a processor to implement the method steps described above.

[0009] According to another aspect of the present application, a computer program product is also provided, which comprises a computer program, wherein the computer program is executable on a processor to implement the method steps described above.

[0010] In the embodiment of the present application, first, the parameter information of the service end is acquired; the parameter information includes the address of the service end, the weight threshold and the preheating time length; then, based on the weight threshold, the preheating time length and the preheated time length, the real-time weight of each service end is determined, and each service end is pre-grouped according to the real-time weight to obtain a pre-grouping result; the pre-grouping result includes a plurality of weight groups; each weight group includes the address of one or more service ends with the same real-time weight; when the calling request is received, the number of available addresses in each weight group is determined by using the dynamic routing result and the pre-grouping result; the dynamic routing result includes one or more available addresses; the weight calculation result is generated by using the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight; finally, the target calling address of the calling request is determined according to the weight calculation result. Through the weight threshold and the preheating time length in the parameter information of the service end, the pre-grouping processing of the address is realized, and by using the pre-grouping method, a large amount of repeated calculation work in the weight calculation process can be moved to the background asynchronous calculation, which is beneficial to remove the heavy workload of traversing the address to obtain the parameter, so that the system can run with extremely low performance loss even when the cluster size is particularly large, and the calculation amount during calling is greatly reduced; after obtaining the pre-grouping result, the weight calculation result is generated by combining the dynamic routing result and the pre-grouping result, and then the load balancing is realized, which provides strong support for preheating calculation in large-scale deployment scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and do not constitute improper limitations to the present application. In the drawings:

[0012] Figure 1 is a flowchart of a load balancing method according to an embodiment of the present application;

[0013] Figure 2 is a schematic diagram of an address screening process according to an embodiment of the present application;

[0014] Figure 3 is a calculation logic diagram of a weight-based random algorithm in a preheating scenario according to an embodiment of the present application;

[0015] Figure 4 is an example diagram of weights calculated at different times in a preheating scenario according to an embodiment of the present application;

[0016] Figure 5 is a schematic diagram of an implementation process based on a traversal addressing scheme according to an embodiment of the present application;

[0017] Figure 6 is a schematic diagram of a binary tree address searching method according to an embodiment of the present application;

[0018] Figure 7 is a schematic diagram of a pre-grouping result according to an embodiment of the present application;

[0019] Figure 8 is a schematic diagram of a weight calculation process according to an embodiment of the present application;

[0020] Figure 9 is a schematic diagram of a storage structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0023] It is considered that in the prior art, weight calculation is widely used in products related to network transmission, in the field of cloud-native microservices, such as brpc (also known as baidu-rpc, which is a remote procedure call network framework developed by Baidu), grpc (which is a high-performance, general-purpose open source RPC framework developed by Google), envoy (which is an open source edge and service proxy for cloud-native applications), etc. There are corresponding logic based on weight-based addressing. The normal weight-based addressing provided by brpc, grpc, and envoy needs to traverse the weight information of each target every time a request is initiated, and perform calculation. When the number of target addresses reaches a certain order of magnitude, the performance loss of each request will be extremely high, occupying a large part of the resources.

[0024] Figure 5 is a schematic diagram of the implementation process of the traversal addressing scheme according to an embodiment of the present application, which can be mainly divided into three processes: parameter reading, single weight calculation, and weight screening. Parameter reading refers to reading the parameter information required for this calculation from the parameter list of the server; single weight calculation refers to calculating the weight of the current address based on the parameters of a single address. For the preheating scenario, the cumulative duration of the server after startup needs to be considered to calculate the current weight; the weight screening stage refers to counting the weight information of all addresses and selecting a server based on a random number. See Figure 3The calculation logic diagram of the weight-based random algorithm in the preheating scenario according to the embodiment of the application is shown. First, the maximum weight, preheating time and other parameters are obtained from the parameter list of the server side. Then, the latest weight information is calculated based on the current time. Finally, a server is selected based on a random number. Referring to Figure 4 The weight example diagram of different time calculation in the preheating scenario according to the embodiment of the application is shown. The diagram includes the weight calculation results at 10 seconds, 50 seconds and 100 seconds preheating time. Figure 3 and Figure 4 The implementation steps of the traversal addressing scheme are shown.

[0025] The traversal addressing scheme needs to calculate all addresses in each request call process, which is an algorithm with a time complexity of O(N) (N is the amount of server addresses). When the magnitude of the server N and the request amount are large, a large amount of computing resources will be consumed during running, which causes the load balancing scheme based on weight preheating to be unable to be widely promoted.

[0026] In the prior art, the LALB (Locality-aware load balancing) algorithm is provided in the brpc. The binary tree search algorithm is used in the weight screening, which optimizes the performance in single call. However, since the weight information is dynamically calculated, the binary tree needs to be reconstructed and maintained all the time. Moreover, the LALB algorithm does not support dynamic routing addressing (removing some addresses before load balancing).

[0027] Figure 6 The addressing mode based on binary tree search according to the embodiment of the application is shown. This mode separates the weight calculation and weight screening processes from the foregoing traversal addressing scheme. The weight of each address is calculated in advance and recorded in the binary tree. The sum of the weight values of the left subtree is recorded in each node of the binary tree. In this way, the search can be completed in O(logN) time during the call. However, this scheme has a great limitation: it cannot work with dynamic routing. Since the binary tree is calculated in advance, the sum of the weight values of the left subtree of each node is also fixed. The nodes cannot be dynamically removed during running. Therefore, this scheme is not suitable for use in complex scenarios.

[0028] In the following embodiments, considering that there are a large number of repetitive operations in runtime traversal address de-parameterization, in order to optimize performance, part of the repetitive calculation work in weight calculation can be pre-processed, so that after once address is taken, the address information is pre-grouped and processed, and the key information is saved, and then the pre-grouping result is obtained to perform calculation, so that the calculation of all addresses is not required in each request call process, so that the weight screening can be started with extremely low performance loss when the cluster size is particularly large. In addition, in the following embodiments, the dynamic routing result is combined to perform load balancing calculation, so that the dynamic routing can be naturally supported.

[0029] Firstly, the involved terms are explained.

[0030] Address: access mode of an RPC (Remote Procedure Call Protocol) service provider.

[0031] Routing: screening of the address of a service provider according to certain rules.

[0032] Load balancing: selecting an optimal address from multiple addresses through a certain algorithm, and achieving maximum utilization of all address resources after the request amount reaches a certain degree.

[0033] In the present embodiment, a load balancing method is provided, Figure 1 is a flowchart of the load balancing method according to the embodiments of the present application, and the method steps involved in Figure 1 will be described below.

[0034] In step S102, parameter information of a service end is acquired; the parameter information includes the address of the service end, a weight threshold, and a preheating time length.

[0035] In this step, the service end pre-registers the related parameter information involved in the call in the registration center, and the consumer end acquires the parameter information of the service end from the registration center, wherein the parameter information includes the address of the service end, the weight threshold, and the preheating time length. The service end can include multiple machines, and each machine corresponds to a group of parameter information. The weight threshold can be used to describe the machine performance of the service end corresponding to the group of parameters, and the machine with a higher weight threshold can execute a larger task amount. Referring to Figure 7For the first full address 10.0.0.1 in the figure, the weight threshold (indicated by weight in the figure) of the machine corresponding to the address is 100, and for the full address 10.0.0.3, the weight threshold of the machine corresponding to the address is 50, which indicates that the machine corresponding to the address 10.0.0.1 is stronger than the machine corresponding to the address 10.0.0.3, and can undertake a relatively larger amount of tasks and has a relatively stronger computing capability. The warm-up duration can be used to describe the duration required for the machine to complete the warm-up, for example, referring to Figure 7 For the first full address 10.0.0.1 in the figure, the warm-up duration (indicated by warmup Time in the figure) of the machine corresponding to the address is 50. According to the address, the access mode of the RPC service provider can be obtained.

[0036] In step S104, based on the weight threshold, the warm-up duration, and the preheated duration, the real-time weight of each service end is determined, and each service end is pre-grouped according to the real-time weight to obtain a pre-grouping result. The pre-grouping result includes a plurality of weight groups, and each weight group includes one or more addresses of service ends with the same real-time weight.

[0037] In this step, the duration required for the warm-up process from starting to ending is the warm-up duration, and the preheated duration is the duration spent from the start of the warm-up to the current time. Based on the proportion of the preheated duration in the warm-up duration, the progress of the warm-up operation can be obtained. The real-time weight of the service end changes with the progress of the warm-up operation, and at the end of the warm-up, the real-time weight of the service end reaches the weight threshold. Therefore, by using the warm-up duration and the preheated duration, the performance occupancy proportion of the service end can be obtained, and in combination with the weight threshold, the performance occupancy of the service end can be described, that is, based on the weight threshold, the warm-up duration, and the preheated duration, the real-time weight of each service end is determined, and the real-time weight can be used to describe the performance occupancy degree of the service end in the warm-up phase. After obtaining the address of the service end, the addresses of the service ends are grouped according to the real-time weight of each address, and the pre-grouping result obtained after grouping is recorded. The pre-grouping result includes a plurality of weight groups, and each weight group includes one or more addresses of service ends with the same real-time weight.

[0038] The weight pre-grouping behavior needs to be continuously performed in the machine startup preheating period in the preheating scenario. The pre-grouping processing stage can be maintained by a background independent thread. In the case of large-scale service end deployment, based on the pre-grouping processing, more than half of the repeated calculation work can be moved to the background asynchronous calculation, reducing the calculation amount and improving the processing efficiency. After performing an operation of obtaining the parameter information of the service end, the real-time weight is calculated based on the obtained parameters, and then the pre-grouping result is obtained, which can be used in the subsequent weight calculation step. Through pre-calculation of the preheating type weight calculation, the situation of multiple iterations of the address at runtime is alleviated, the frequency of the address acquisition of the consumer is reduced, and the calculation amount during the calling is greatly reduced.

[0039] In step S106, when receiving the calling request, the number of available addresses in each weight group is determined by using the dynamic routing result and the pre-grouping result. The dynamic routing result includes one or more available addresses.

[0040] In this step, when receiving the calling request, the target calling address allocated for the calling request is started. Before load balancing, the address information of the service end is filtered to obtain a dynamic routing result by dynamic routing. The dynamic routing result excludes the address information of the service end that is unavailable or does not meet the use condition. The dynamic routing result includes one or more available addresses. The pre-grouping result groups all the addresses of the service end to obtain a plurality of weight groups. Based on the dynamic routing result and the plurality of weight groups, the unavailable addresses in the weight groups are excluded, and then the number of available addresses in each weight group is obtained.

[0041] In the above step, the dynamic routing step excludes a part of unavailable addresses. In this step, based on the dynamic routing result, the unavailable addresses in each weight group of the pre-grouping result are also excluded. The available addresses in the pre-grouping result are filtered based on the dynamic routing result, which naturally supports dynamic routing. This step can quickly filter out the available addresses, and the complexity of the iteration calculation scheme is compressed from O(N) to close to O(1), where N is the number of service end machines, so that the weight filtering can be started with extremely low performance loss even when the cluster size is particularly large.

[0042] In step S108, the weight calculation result is generated by using the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight.

[0043] In this step, since each weight group has one or more addresses with the same real-time weight, for the same weight group, the number of available addresses in the same weight group and the weight value of the group can be used to generate a group weight value. The sum of the group weight values of all weight groups is obtained, and the weight calculation result is obtained.

[0044] In the above step, the weight calculation result is generated by using the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight, avoiding complete pre-warming weight calculation on all service ends, and improving the weight calculation efficiency in the pre-warming scene.

[0045] In specific implementation, the weight calculation process diagram shown in Figure 8 may be referred to, the number of available addresses in the weight group with a weight value of 20 is 2, and the group weight value of the group is calculated to be 40 based on the weight value 20 and the number of available addresses 2. According to the same method, the total weight value of each group is calculated. As shown in Figure 8 , the result value of the dynamic routing is left with 3 addresses, the group with a weight of 20 is left with only 2 addresses after comparison, the group with a weight of 10 does not exist after comparison, and the group with a weight of 5 is left with only 1 address after comparison, and then the total weight value of each group is 40, 0 and 5 respectively, and the total weight value of the dynamic routing result is 45, that is, the weight calculation result is 45.

[0046] Step S110, determining the target calling address of the calling request according to the weight calculation result.

[0047] In this step, the generation of the weight calculation result is based on the pre-grouping processing result, and the pre-grouping processing result is obtained based on the weight threshold, the pre-warming time and the pre-warming time, therefore, the target calling address obtained based on the weight calculation result can be applied in the pre-warming scene, and provides strong support for pre-warming calculation.

[0048] It should be noted that the weight-based random algorithm extension can realize the service end pre-warming function, the purpose of which is to reduce the weight when the service end is just started, and only allow small flow to enter, and then put the full flow into the service end when the optimization processing of the service end is completed (such as connection pool, Java C2 compilation, etc.), so as to reduce the influence caused by the start lag.

[0049] The application provides a load balancing method, which first acquires parameter information of a service end; the parameter information comprises an address of the service end, a weight threshold and a preheating time length; then, based on the weight threshold, the preheating time length and a preheating time length, real-time weights of each service end are determined, and each service end is pre-grouped according to the real-time weights to obtain a pre-grouping result; the pre-grouping result comprises a plurality of weight groups; each weight group comprises addresses of one or more service ends with the same real-time weight; when a calling request is received, the number of available addresses in each weight group is determined by using a dynamic routing result and the pre-grouping result; the dynamic routing result comprises one or more available addresses; a weight calculation result is generated by using the real-time weight of each weight group and the number of available addresses corresponding to the real-time weight; finally, a target calling address of the calling request is determined according to the weight calculation result. By using the weight threshold and the preheating time length in the parameter information of the service end, the application realizes pre-grouping processing of the address, and by using the pre-grouping method, a large amount of repeated calculation work in the weight calculation process can be moved to the background asynchronous calculation, which is beneficial to removing the heavy workload of traversing the address to obtain the parameter, so that the system can run with extremely low performance loss even when the cluster size is particularly large, and the calculation amount during calling is greatly reduced; after obtaining the pre-grouping result, the weight calculation result is generated by combining the dynamic routing result and the pre-grouping result, and then the load balancing is realized, which provides strong support for preheating calculation in a large-scale deployment scenario.

[0050] In order to further improve the efficiency of weight calculation, as an optional embodiment, the number of available addresses in each weight group is determined by using the dynamic routing result and the pre-grouping result, and the following steps are performed:

[0051] If the dynamic routing result and the weight group have the same address, the same address is taken as an available address; the number of available addresses in each weight group is counted.

[0052] In this step, the dynamic routing result is compared with each weight group respectively, the same address is taken as an available address, and the addresses in each weight group that are not the same are removed; then, the number of available addresses in each weight group is counted.

[0053] In the specific implementation of this step, for example, refer to the weight calculation process diagram according to the embodiment of the application shown in Figure 8 The dynamic routing result is compared with the weight group with a weight value of 20, and the addresses corresponding to the dashed box with a weight of 20 in the figure are filtered out. When calling, the result of the dynamic routing is compared with the weight result of each group, and the number of current remaining addresses in each group, i.e. the number of available addresses, can be quickly obtained.

[0054] In an optional embodiment, if the dynamic routing result and the weight group have the same address, the same address is regarded as an available address, and the following steps can be performed:

[0055] The addresses of the service end are sorted to obtain sorting information; according to the sorting information, the presence of the addresses of the service end in the dynamic routing result and each weight group is marked by using binary bits; if the marked dynamic routing result and the marked weight group have the same binary value at the same sorting position, the address of the weight group at the sorting position is regarded as an available address.

[0056] In this step, the addresses of the service end are sorted according to a preset sorting rule, for example, the addresses can be sorted according to the serial numbers of the addresses or sorted according to a random distribution order. After sorting, the sorting information obtained is recorded, and the dynamic routing result is marked according to the order of the sorting information by using binary bits to obtain a binary marking result, for example, if the address exists, it is marked as “1”, and if the address does not exist, it is marked as “0”. The addresses in each weight group are also marked by using binary bits according to the order of the sorting information to obtain a binary marking result, and the marking method is the same as above and will not be repeated here. Finally, the marked dynamic routing result and the marked weight group are compared according to the sorting position to obtain an available address, that is, if the marked dynamic routing result and the marked weight group have the same binary value at the same sorting position, the address of the weight group at the sorting position is regarded as an available address. In an optional embodiment, if the marked dynamic routing result and the marked weight group have the same binary value at the same sorting position, the address of the weight group at the sorting position is regarded as an available address, including: performing an AND operation on the marked dynamic routing result and each marked weight group to determine whether the binary values at the same sorting position are the same.

[0057] In this step, the runtime uses bit operation to quickly compare to obtain the real available address under each group, which naturally supports dynamic routing calculation. Since the bit operation is used for comparison in the embodiment of the application, the time complexity is compressed from O(N) to a simple CPU bottom-level AND instruction when obtaining the total weight value, the actual time consumption is close to O(1), and the influence of the cluster size is small, thereby providing strong support for preheating calculation in large-scale deployment scenarios.

[0058] In the specific implementation of this step, refer to Figure 9The diagram shown is a storage structure diagram according to an embodiment of this application. This diagram is the storage structure corresponding to the solution of this application. In the solution of this application, binary bits are used to mark whether each address exists. For the case of 5 addresses, if the binary storage bit is 00100, it means that only the address where the 3rd bit is located exists, and the others are filtered out.

[0059] by Figure 9 For example, the original dynamic routing result has three bits, corresponding to the 1st, 4th, and 5th address bits, with a corresponding binary storage value of 10011. Similarly, the original address result for a packet cache with a weight of 20 also has three bits, corresponding to the 1st, 2nd, and 5th address bits, with a corresponding binary storage value of 11001. Likewise, the packet storage value for a weight of 10 is 00100, indicating that the 3rd address exists in the current packet; the packet storage value for a weight of 5 is 00010, indicating that the 4th address exists in the current packet. The dynamic routing result is marked as 10011, meaning that the routing result only contains the 1st, 4th, and 5th addresses.

[0060] After representing each address group using binary bits, the intersection of two result groups can be quickly obtained through a binary AND operation. For example, ANDing 10011 and 11001 yields 10001, indicating that bits 1 and 5 are common. Based on this, performing a bitwise AND operation between the binary value of each group and the dynamic routing result yields the remaining address information for each group under the current dynamic routing result. Finally, the actual number of addresses currently existing under each group's weight can be determined.

[0061] After obtaining the number of addresses, the total weight value of the group can be obtained based on the weight attribute of the group itself, and then the total weight value of all groups can be obtained.

[0062] The main task of the weight pre-grouping stage is to calculate the real-time weight of each address. In order to obtain the weight information updated during machine warm-up, as an optional implementation, the real-time weight of each server is determined based on the weight threshold, the warm-up duration, and the pre-warm-up duration. The server is then pre-grouped according to the real-time weight, and the following steps are performed: calculate the weight coefficient based on the warm-up duration and the pre-warm-up duration; calculate the real-time weight of each address of the server using the weight coefficient and the weight threshold; and add addresses with the same real-time weight to the same weight group.

[0063] In this step, the preheating duration can represent the progress of the preheating phase. A weighting coefficient is calculated based on the preheating duration and the preheated duration. The specific method for calculating the weighting coefficient can be selected according to actual needs, and this embodiment does not impose specific limitations. In specific implementation, the ratio of the preheated duration to the total preheating duration can be used as the weighting coefficient. It should be noted that the value of the weighting coefficient does not exceed 1. Then, the real-time weight of each address on the server is calculated using the weighting coefficient and the weighting threshold; addresses with the same real-time weight are added to the same weighting reassembly, thereby obtaining multiple weighting reassemblies.

[0064] For specific implementation details, please refer to [link / reference]. Figure 7 The diagram shown is a pre-grouping result illustration according to an embodiment of this application. Figure 7 For the weight distribution at a warm-up time of 10 seconds, there are 3 addresses with a weight calculation result of 20, 1 address with a weight calculation result of 10, and 1 address with a weight calculation result of 5. This step uses a weight pre-grouping method, moving more than half of the repetitive calculation work of weight calculation to asynchronous calculation in the background. This helps to remove the hotspots of cumbersome address traversal and parameter retrieval during runtime, and can effectively reduce the amount of computation.

[0065] In this embodiment, the weight pre-grouping behavior needs to be continuously executed during the machine startup warm-up cycle. However, for large-scale deployment clusters, the weights of individual machines do not require particularly high accuracy. Therefore, the frequency of weight pre-computation is not high, and the timeliness requirement is also low. (Compared to the LALB algorithm in bRPC, the LALB algorithm cannot be deployed in large-scale clusters due to its small time window and the fact that cluster traffic scheduling depends on the accuracy of the weights.) Therefore, to further reduce performance loss, as an optional implementation method, the method can also perform the following steps:

[0066] When the warm-up time of each server exceeds the warm-up time, if a call request is received, a weight calculation result is generated based on the dynamic routing result using a random algorithm.

[0067] In this step, if the preheating time for each server exceeds the preheating duration, it indicates that the preheating phase has ended. After the preheating phase ends, when the background calculation thread finds that all address weights are consistent, i.e., the real-time weights of all addresses have reached the weight threshold and no longer change, this embodiment automatically degenerates to a normal random algorithm. During the call, the grouping results are no longer compared; instead, weight calculation is performed directly based on the dynamic routing results and the random algorithm, thereby further reducing performance overhead. This embodiment periodically checks the weight results, and when the global weights are the same, quickly skips the weight calculation logic to avoid unnecessary calculations.

[0068] After obtaining the weight calculation result, as an optional implementation, the target calling address of the calling request is determined according to the weight calculation result, and the following steps are performed:

[0069] A weight value range is generated by using the weight calculation result, a target weight is determined in the weight value range by using a target random algorithm, and the target calling address of the calling request is determined according to the real-time weight in the pre-grouping result and the target weight.

[0070] In this step, a weight value range is generated by using the weight calculation result, the weight calculation result can be used as one endpoint of the weight value range, and 0 value is used as another endpoint. After the weight value range is determined, a target weight is determined in the weight value range by using a target random algorithm. For example, after obtaining the weight calculation result as 45, a value of [0, 45) can be obtained by a random number, so as to obtain the target weight selected this time. Then, the address information corresponding to the weight can be obtained by using the weight reduction mechanism, and then the target calling address of the calling request is determined according to the real-time weight in the pre-grouping result and the target weight.

[0071] In order to further improve the address screening efficiency, as an optional implementation, the target calling address of the calling request is determined according to the real-time weight in the pre-grouping result and the target weight, and the following steps are performed:

[0072] The address information of the server is sorted according to the real-time weight in the pre-grouping processing result, and an address sorting result is obtained. The weight values of one or more address information of the server are summed according to the address sorting result, and a sum result is obtained. When the sum result is not less than the target weight, the address information corresponding to the calculation result is determined as the target calling address.

[0073] In this step, since the real-time weight of each address is recorded in the pre-grouping processing result, the cumulative real-time weight is calculated according to the sorting result. When the sum result is not less than the target weight, the address information whose weight value is added to the sum result is determined as the target calling address.

[0074] In order to further improve the address screening efficiency, as an optional implementation, the target calling address of the calling request is determined according to the real-time weight in the pre-grouping result and the target weight, and the following steps are performed:

[0075] The real-time weight of each address in the pre-grouping result is recorded by using a binary tree. The target calling address of the calling request is calculated based on the binary tree and the target weight.

[0076] In this step, the sum of the weight values of the left sub-tree is recorded in each node of the binary tree, so that the search can be completed in O(logN) time when called, improving the processing efficiency.

[0077] In order to improve the calculation efficiency of the weight calculation result, as a preferred embodiment, the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight are used to generate the weight calculation result, which can be executed according to the following steps: the product of the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight is used to calculate the group weight value; sum all the group weight values to obtain the weight calculation result.

[0078] In this step, the product can quickly calculate the group weight value of each weight group, and then sum the group weight values to obtain the weight calculation result.

[0079] In order to achieve maximum utilization of all address resources after the request reaches a certain level, after determining the target calling address of the calling request according to the weight calculation result, the following steps can be performed: calling the server according to the target calling address.

[0080] The application provides a load balancing method, which can be applied in related products such as Dubbo3 (a microservice development framework), MSE (Microservices Engine), microservice governance, gateway, SAE (Serverless AppEngine), ASM (Alibaba Cloud Service Mesh), CSB (Cloud Service Bus), EDAS (Enterprise Distributed Application Service), and HSF3 (a microservice development framework, full name High-speed Service Framework). The method splits the calculation of the weight into a weight pre-grouping stage and a weight comparison stage, and the weight pre-grouping stage is independently maintained by a background thread. The method pre-groups the weight, quickly matches the weight of the available server based on bit operation, and in the computer weight stage, the complexity of the traversal calculation scheme is compressed from O(N) to close to O(1), so that the weight screening can be started with extremely low performance loss even when the cluster size is particularly large. In addition, the method makes it possible to calculate the normal weight, and the method supports dynamic calculation of the route and has the ability to land in complex scenarios.

[0081] In this embodiment, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method in the above embodiments when executing the computer program.

[0082] The above program can be run in the processor, or can also be stored in the memory (or called computer readable medium), the computer readable medium includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. Information can be computer readable instructions, data structure, program module or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this paper, computer readable medium does not include transitory computer readable medium, such as modulated data signal and carrier wave.

[0083] These computer programs can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate computer implemented processing, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 The flow or multiple flows and / or the functions specified in the block Figure 1 The steps of one or more blocks or multiple blocks, corresponding to different steps can be realized by different modules.

[0084] The embodiment provides the device. The device is called a load balancing device, and the device comprises a parameter acquisition module, a pre-grouping module, a comparison module, a weight calculation module and a load balancing module. The parameter acquisition module is used for acquiring parameter information of a service end, and the parameter information comprises an address of the service end, a weight threshold and a preheating time length. The pre-grouping module is used for determining real-time weights of the service ends based on the weight threshold, the preheating time length and an already preheated time length, performing weight pre-grouping on the service ends according to the real-time weights, and obtaining a pre-grouping result. The pre-grouping result comprises a plurality of weight groups. Each weight group comprises addresses of one or more service ends with the same real-time weight. The comparison module is used for determining a number of available addresses in each weight group by using a dynamic routing result and the pre-grouping result. The dynamic routing result comprises one or more available addresses. The weight calculation module is used for generating a weight calculation result by using the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight. The load balancing module is used for determining a target calling address of the calling request according to the weight calculation result.

[0085] The system or the device is used for realizing the functions of the method in the above-mentioned embodiments. Each module in the system or the device corresponds to each step in the method, and has been described in the method and will not be described here again.

[0086] Optionally, the number of available addresses in each weight group is determined by using the dynamic routing result and the pre-grouping result, and the method comprises: if the same address exists in the dynamic routing result and the weight group, the same address is taken as an available address; and the number of available addresses in each weight group is counted.

[0087] Optionally, if the same address exists in the dynamic routing result and the weight group, the same address is taken as an available address, and the method comprises: sorting the addresses of the service ends to obtain sorting information; using binary bits to mark the existence states of the addresses of the service ends in the dynamic routing result and in each weight group according to the sorting information; and if the binary values of the same sorting positions in the marked dynamic routing result and the marked weight group are the same, the address of the same sorting position in the weight group is taken as an available address.

[0088] Optionally, if the binary values of the same sorting positions in the marked dynamic routing result and the marked weight group are the same, the address of the same sorting position in the weight group is taken as an available address, and the method comprises: performing an AND operation on the marked dynamic routing result and each marked weight group respectively to determine whether the binary values of the same sorting positions are the same.

[0089] Optionally, based on the weight threshold, the pre-warming duration and the pre-warming duration of each service end, the real-time weight of each service end is determined, and each service end is pre-grouped according to the real-time weight, comprising: calculating a weight coefficient according to the pre-warming duration and the pre-warming duration; calculating the real-time weight of each address of the service end by using the weight coefficient and the weight threshold; adding the addresses with the same real-time weight to the same weight group.

[0090] Optionally, it further comprises: when the pre-warming duration of each service end is greater than the pre-warming duration, if a calling request is received, a weight calculation result is generated based on the dynamic routing result by using a random algorithm.

[0091] Optionally, the target calling address of the calling request is determined according to the weight calculation result, comprising: generating a weight value range by using the weight calculation result; determining a target weight in the weight value range by using a target random algorithm; determining the target calling address of the calling request according to the real-time weight in the pre-grouping result and the target weight.

[0092] Optionally, the target calling address of the calling request is determined according to the real-time weight in the pre-grouping result and the target weight, comprising: sorting the address information of the service end according to the real-time weight in the pre-grouping processing result to obtain an address sorting result; summing the real-time weights of one or more addresses of the service end according to the address sorting result to obtain a sum result; when the sum result is not less than the target weight, the address information corresponding to the calculation result is determined as the target calling address.

[0093] Optionally, the target calling address of the calling request is determined according to the real-time weight in the pre-grouping result and the target weight, comprising: recording the real-time weight of each address in the pre-grouping result by using a binary tree; calculating the target calling address of the calling request based on the binary tree and the target weight.

[0094] Optionally, the weight calculation result is generated by using the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight, comprising: calculating a group weight value by using the product of the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight; summing all the group weight values to obtain the weight calculation result.

[0095] Optionally, after the target calling address of the calling request is determined according to the weight calculation result, it further comprises: calling the service end according to the target calling address.

[0096] The above embodiments solve the problem that when the magnitude of the service end and the request amount are large, the running consumes a large amount of computing resources in the prior art under the preheating scenario. The above embodiments of the present application realize pre-grouping processing of address information through the address, weight threshold and preheating duration in the service end parameter information. In the pre-grouping manner, a large amount of repeated calculation work in the weight calculation process can be moved to background asynchronous calculation, which is beneficial to remove the heavy workload of traversing address to obtain parameters at runtime, so that the system can run with extremely low performance loss even when the cluster size is particularly large, greatly reducing the calculation amount when calling; after obtaining the pre-grouping result, the weight calculation result is generated in combination with the dynamic routing result and the pre-grouping result, and then load balancing is realized, thereby providing strong support for preheating calculation in a large-scale deployment scenario.

[0097] In the embodiment, a computer readable storage medium is further provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method in the above embodiment.

[0098] In the embodiment, a computer program product is further provided, and the computer program product includes a computer program. The computer program is executed by a processor to implement the method in the above embodiment.

[0099] The above is only an embodiment of the present application and is not used to limit the present application. The present application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement and the like within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A load balancing method, characterized by, The application comprises: acquiring parameter information of a service end; the parameter information comprises an address of the service end, a weight threshold value and a preheating time length; based on the weight threshold value, the preheating time length and a preheated time length, determining real-time weights of each service end, and pre-grouping the service ends according to the real-time weights to obtain a pre-grouping result; the pre-grouping result comprises a plurality of weight groups; each weight group comprises addresses of one or more service ends with the same real-time weight, and a processing stage of the pre-grouping is maintained by an independent thread of a background; when a calling request is received, determining a number of available addresses in each weight group by using a dynamic routing result and the pre-grouping result; the dynamic routing result comprises one or more available addresses; generating a weight calculation result by using the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight; determining a target calling address of the calling request according to the weight calculation result.

2. The method of claim 1, wherein, determining a number of available addresses in each weight group by using a dynamic routing result and the pre-grouping result, comprises: if the dynamic routing result and the weight group have the same address, the same address is taken as an available address; counting the number of available addresses in each weight group.

3. The method of claim 2, wherein, if the dynamic routing result and the weight group have the same address, the same address is taken as an available address, comprising: sorting addresses of the service end to obtain sorting information; according to the sorting information, using binary bits to mark the presence state of the addresses of the service end in the dynamic routing result and each weight group respectively; if the binary values of the same sorting position of the marked dynamic routing result and the marked weight group are the same, the address of the weight group at the same sorting position is taken as an available address.

4. The method of claim 3, wherein, if the binary values of the same sorting position of the marked dynamic routing result and the marked weight group are the same, the address of the weight group at the same sorting position is taken as an available address, comprising: performing an AND operation on the marked dynamic routing result and each marked weight group respectively to determine whether the binary values of the same sorting position are the same.

5. The method of claim 1, wherein, based on the weight threshold value, the preheating time length and a preheated time length, determining real-time weights of each service end, and pre-grouping the service ends according to the real-time weights, comprises: calculating a weight coefficient according to the preheating time length and the preheated time length; calculating real-time weights of each address of the service end by using the weight coefficient and the weight threshold value; adding the addresses with the same real-time weight to the same weight group.

6. The method of claim 1, wherein, further comprising: when the preheated time length of each service end is greater than the preheating time length, if a calling request is received, generating a weight calculation result based on the dynamic routing result by using a random algorithm.

7. The method of claim 1, wherein, determining a target calling address of the calling request according to the weight calculation result, comprises: generating a weight value range by using the weight calculation result; determining a target weight in the weight value range by using a target random algorithm; determining the target calling address of the calling request according to the real-time weight in the pre-grouping result and the target weight.

8. The method of claim 7, wherein, According to the real-time weight in the pre-grouping result and the target weight, a target call address of the call request is determined, including: According to the real-time weight in the pre-grouping result, address information of the server is sorted to obtain a sorting result of the address information; According to the sorting result of the address information, real-time weights of one or more addresses of the server are summed to obtain a sum result; When the sum result is not less than the target weight, address information corresponding to the calculation result is determined as the target call address.

9. The method of claim 7, wherein, According to the real-time weight in the pre-grouping result and the target weight, a target call address of the call request is determined, including: The real-time weight of each address in the pre-grouping result is recorded by using a binary tree; Based on the binary tree and the target weight, the target call address of the call request is calculated.

10. The method of claim 1, wherein, The real-time weight of the weight group and the number of available addresses corresponding to the real-time weight are used to generate a weight calculation result, including: The product of the real-time weight of the weight group and the number of available addresses corresponding to the real-time weight is used to calculate a grouping weight value; All the grouping weight values are summed to obtain a weight calculation result.

11. The method according to any one of claims 1-10, characterized in that, After the target call address of the call request is determined according to the weight calculation result, the method further includes: According to the target call address, the server is called.

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method steps of any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps of any one of claims 1 to 11.

14. A computer program product, characterised in that, The computer program product includes a computer program, and the computer program is executed by the processor to implement the method steps of any one of claims 1 to 11.

Images