A load balancing method and device

Abstract

The present invention discloses a load balancing method and device, and relates to the field of computer technology. A specific implementation of the method includes: obtaining a first performance indicator of a server in a server cluster, selecting a server from the server cluster according to the first performance indicator and a set screening rule, and obtaining a first server subset; obtaining a second performance indicator of a server in the first server subset, selecting a server from the first server subset according to the second performance indicator and a set indicator threshold, and obtaining a second server subset; calculating the indicator weighted value of the server in the second server subset according to the weight information and the second performance indicator configured for the server, and sending a network request to the server corresponding to the minimum indicator weighted value. This implementation method perceives the server performance and real-time network status by obtaining the first performance indicator and the second performance indicator, and then distributes network requests to avoid request backlogs, has strong adaptability, and makes resource utilization more rational.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a load balancing method and device. Background Art

[0002] In order to improve the throughput and concurrency of the entire system, load balancing technology is needed to evenly distribute network requests to each server in the server cluster. The implementation principle of existing load balancing technology is generally: using a certain calculation method to select a suitable server from the server cluster and return it. For example, using a random algorithm to select a server from the server cluster and return it.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The existing load balancing technology distributes network requests without knowing the performance of each server and the network status, which easily causes request backlogs, uneven CPU (Central Processing Unit) utilization of the entire server cluster, and low OPS (Operation per second) that the server cluster can bear. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a load balancing method and device, which perceives server performance and real-time network status by acquiring a first performance indicator and a second performance indicator, screens servers, and then allocates network requests to avoid request backlogs. It has strong adaptability and more rational resource utilization.

[0006] To achieve the above objective, according to one aspect of an embodiment of the present invention, a load balancing method is provided.

[0007] A load balancing method according to an embodiment of the present invention includes: obtaining a first performance indicator of a server in a server cluster, selecting a server from the server cluster according to the first performance indicator and a set screening rule, and obtaining a first server subset; wherein the first performance indicator represents request processing efficiency; obtaining a second performance indicator of a server in the first server subset, selecting a server from the first server subset according to the second performance indicator and a set indicator threshold, and obtaining a second server subset; wherein the second performance indicator represents request processing delay; calculating the indicator weighted value of the server in the second server subset according to the weight information configured for the server and the second performance indicator, and sending the network request to the server corresponding to the minimum indicator weighted value.

[0008] Optionally, the second performance indicator is a TP indicator or an average response time; based on the second performance indicator and a set indicator threshold, a server is selected from the first server subset to obtain a second server subset, including: determining whether the second performance indicator of the server in the first server subset is greater than or equal to the set indicator threshold; filtering servers whose second performance indicator is greater than or equal to the indicator threshold from the first server subset to obtain the second server subset.

[0009] Optionally, based on the weight information configured for the server and the second performance indicator, the indicator weighted value of the server in the second server subset is calculated, including: determining whether the server in the second server subset is configured with weight information; if the server is not configured with the weight information, using the second performance indicator as the indicator weighted value of the server; if the server is configured with the weight information, using the multiplication result of the weight information and the second performance indicator as the indicator weighted value of the server.

[0010] Optionally, the first performance indicator is the number of call failures, throughput or average throughput; the screening rule includes a first sub-rule and a second sub-rule; according to the first performance indicator and the set screening rule, a server is selected from the server cluster to obtain a first server subset, including: according to the first performance indicator and the first sub-rule, a server is selected from the server cluster to obtain a subcluster; calculating the proportion of servers in the subcluster that meet the set benchmark value of the first performance indicator, and comparing the proportion with a set ratio threshold; according to the comparison result, the subcluster is used as the first server subset, or according to the third performance indicator and the second sub-rule, a server is selected from the subcluster to obtain the first server subset; wherein the third performance indicator represents the server load condition.

[0011] Optionally, based on the comparison result, the subcluster is used as the first server subset, or based on the third performance indicator and the second sub-rule, servers are selected from the subcluster to obtain the first server subset, including: if the proportion is less than the proportion threshold, the subcluster is used as the first server subset; if the proportion is greater than or equal to the proportion threshold, the third performance indicator of the servers in the subcluster is obtained, and based on the third performance indicator and the second sub-rule, servers are selected from the subcluster to obtain the first server subset.

[0012] Optionally, the third performance indicator is the number of concurrent requests or the number of user requests; the first sub-rule is: the first M servers with high request processing efficiency in the server cluster; wherein M is a custom integer, or the product of the total number of servers in the server cluster and a set first ratio; the second sub-rule is: the first N servers with low load in the subcluster; wherein N is a custom integer, or the product of the total number of servers in the subcluster and a set second ratio.

[0013] Optionally, the method also includes: looping the acquisition and selection steps and the calculation and sending steps, so that the network requests sent to the servers with high load in the server cluster gradually decrease, and the network requests sent to the servers with low load in the server cluster gradually increase; wherein the acquisition and selection step is a step of acquiring the second performance indicator and selecting a server from the first server subset; the calculation and sending step is a step of calculating the indicator weighted value and sending the network request to the server corresponding to the minimum indicator weighted value.

[0014] To achieve the above objective, according to another aspect of an embodiment of the present invention, a load balancing device is provided.

[0015] A load balancing device according to an embodiment of the present invention includes: a first selection module, used to obtain a first performance indicator of a server in a server cluster, and select a server from the server cluster according to the first performance indicator and a set screening rule to obtain a first server subset; wherein the first performance indicator represents request processing efficiency; a second selection module, used to obtain a second performance indicator of a server in the first server subset, and select a server from the first server subset according to the second performance indicator and a set indicator threshold to obtain a second server subset; wherein the second performance indicator represents request processing latency; a request sending module, used to calculate an indicator weighted value of the servers in the second server subset according to the weight information configured for the server and the second performance indicator, and send the network request to the server corresponding to the minimum indicator weighted value.

[0016] Optionally, the second performance indicator is a TP indicator or an average response time; the second selection module is also used to: determine whether the second performance indicator of the server in the first server subset is greater than or equal to a set indicator threshold; and filter the servers whose second performance indicator is greater than or equal to the indicator threshold from the first server subset to obtain the second server subset.

[0017] Optionally, the request sending module is also used to determine whether the server in the second server subset is configured with weight information; if the server is not configured with the weight information, the second performance indicator is used as the indicator weighted value of the server; if the server is configured with the weight information, the multiplication result of the weight information and the second performance indicator is used as the indicator weighted value of the server.

[0018] Optionally, the first performance indicator is the number of call failures, throughput or average throughput; the screening rule includes a first sub-rule and a second sub-rule; the first selection module is also used to select a server from the server cluster according to the first performance indicator and the first sub-rule to obtain a sub-cluster; calculate the proportion of servers in the sub-cluster that meet the set benchmark value of the first performance indicator, and compare the proportion with a set ratio threshold; based on the comparison result, use the sub-cluster as the first server subset, or select a server from the sub-cluster according to a third performance indicator and the second sub-rule to obtain the first server subset; wherein the third performance indicator represents the server load condition.

[0019] Optionally, the first selection module is further used to, if the proportion is less than the proportion threshold, use the subcluster as the first server subset; if the proportion is greater than or equal to the proportion threshold, obtain a third performance indicator of the servers in the subcluster, and select a server from the subcluster according to the third performance indicator and the second sub-rule to obtain the first server subset.

[0020] Optionally, the third performance indicator is the number of concurrent requests or the number of user requests; the first sub-rule is: the first M servers with high request processing efficiency in the server cluster; wherein M is a custom integer, or the product of the total number of servers in the server cluster and a set first ratio; the second sub-rule is: the first N servers with low load in the subcluster; wherein N is a custom integer, or the product of the total number of servers in the subcluster and a set second ratio.

[0021] Optionally, the device also includes: a loop processing module, which is used to cyclically execute the acquisition and selection steps and the calculation and sending steps, so that the network requests sent to the servers with high loads in the server cluster gradually decrease, and the network requests sent to the servers with low loads in the server cluster gradually increase; wherein the acquisition and selection step is a step of acquiring the second performance indicator and selecting a server from the first server subset; the calculation and sending step is a step of calculating the indicator weighted value and sending the network request to the server corresponding to the minimum indicator weighted value.

[0022] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0023] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a load balancing method according to an embodiment of the present invention.

[0024] To achieve the above objective, according to another aspect of the embodiments of the present invention, a computer readable medium is provided.

[0025] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, a load balancing method according to an embodiment of the present invention is implemented.

[0026] One embodiment of the above invention has the following advantages or beneficial effects:

[0027] By obtaining the first performance indicator and the second performance indicator, the server performance and real-time network status are sensed, and the servers are screened, and then network requests are allocated to avoid request backlogs. The system has strong adaptability and more rational resource utilization. The TP indicator or average response time is used to screen out servers with small request processing delays, so as to give priority to allocating more network requests to these servers, avoid request backlogs, and make rational use of resources. Based on the weight setting, the indicator weighted value of the server is calculated, and then network requests are allocated according to the contribution of the server to the cluster.

[0028] Use the number of call failures, throughput or average throughput to evaluate the server's request processing capability, and select servers based on server load conditions to prioritize network requests to servers with strong processing capabilities and low loads, which can better evaluate cluster performance and make rational use of resources. When the server's request processing efficiency is high, select servers with low loads based on load conditions, so that network requests can be preferentially allocated to servers with low loads in the future.

[0029] By cyclically acquiring the second performance indicator and allocating network requests, servers with high load in the cluster are gradually allocated to fewer network requests, and servers with low load are gradually allocated to more network requests. Resource utilization is more reasonable, ensuring the overall balance of the cluster, increasing the tolerable OPS, and enhancing the adaptability to network emergencies.

[0030] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0032] Figure 1 is a schematic diagram of main steps of a load balancing method according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the main flow of the load balancing method according to an embodiment of the present invention;

[0034] Figure 3 is a flowchart of selecting a server from a sub-cluster in an embodiment of the present invention;

[0035] Figure 4 is a flowchart for calculating the weighted value of an indicator in an embodiment of the present invention;

[0036] Figure 5 is a comparison diagram of the effects before and after using the load balancing method of an embodiment of the present invention;

[0037] Figure 6 is a schematic diagram of main modules of a load balancing device according to an embodiment of the present invention;

[0038] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0039] Figure 8 It is a schematic diagram of the structure of a computer device suitable for implementing the electronic device of the embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0041] Figure 1 FIG. 1 is a schematic diagram of the main steps of the load balancing method according to an embodiment of the present invention. Figure 1 As shown, the load balancing method of the embodiment of the present invention mainly includes the following steps:

[0042] Step S101: Obtain a first performance indicator of a server in a server cluster, and select a server from the server cluster according to the first performance indicator and a set screening rule to obtain a first server subset. The first performance indicator represents the request processing efficiency, such as the number of call failures, throughput, average throughput, etc. The number of call failures is the number of call failures in K calls (such as 100 times, 150 times, etc.), the throughput is the number of requests that the cluster can process per unit time, and the average throughput is the average value of the throughput over a period of time.

[0043] The servers in the server cluster are traversed, and the first performance index of the servers is detected to obtain the request processing efficiency of each server. Then, a number of servers are selected from the server cluster according to the first performance index and the screening rule, and the selected servers constitute the first server subset. The screening rule is used to select servers with high request processing efficiency or low current load from the server cluster.

[0044] Step S102: Obtain a second performance indicator of the servers in the first server subset, and select a server from the first server subset according to the second performance indicator and the set indicator threshold to obtain a second server subset. The second performance indicator characterizes the request processing delay, such as a TP (Top Percentile) indicator, an average response time, etc. TP indicators include TP99, TP999, etc., which refer to the response time of 99% and 99.9% of the servers in the cluster respectively; the average response time refers to the average value of the response time over a period of time. Compared with TP99, TP999 can better reflect the overall cluster performance.

[0045] In this step, the second performance index of the servers in the first server subset is detected to obtain the request processing delay of each server. Then, based on the second performance index and the index threshold, the servers whose second performance index is less than or equal to the index threshold are selected from the first server subset, and the selected servers constitute the second server subset. The index threshold can be customized according to the needs.

[0046] Step S103: Calculate the index weighted values ​​of the servers in the second server subset according to the weight information configured for the server and the second performance index, and send the network request to the server corresponding to the minimum index weighted value. Determine whether the servers in the second server subset are configured with weight information. If the weight information is not configured, it means that the contributions of each server to the cluster are the same, and the network request is preferentially sent to the server corresponding to the minimum second performance index.

[0047] If weight information is configured, indicating that the contribution of each server to the cluster varies, the weighted value of the indicator is calculated according to the weight, and the network request is preferentially sent to the server corresponding to the minimum weighted value of the indicator. This embodiment senses the server performance and real-time network status by acquiring the first performance indicator and the second performance indicator, and then dynamically adjusts the number of network requests allocated to each server according to the server performance and the real-time network status, avoiding request backlog, having strong adaptability to network conditions, and making resource utilization more rational.

[0048] Figure 2 FIG. 1 is a schematic diagram of the main flow of the load balancing method according to an embodiment of the present invention. Figure 2 As shown, the load balancing method of the embodiment of the present invention mainly includes the following steps:

[0049] Step S201: traverse and obtain the servers in the server cluster and the first performance index of each server. For the server cluster that needs to be load balanced, traverse the servers in the cluster to obtain the first performance index of each server. In the embodiment, the server is a service provider (Provider), and the service provider is the party that provides services as a server in an RPC (Remote Procedure Call Protocol) call.

[0050] Since the first performance indicator can directly reflect the overall pressure load of a server cluster, this indicator is processed first. Taking the first performance indicator as the number of call failures as an example, it can be obtained through the built-in call success rate counter of each server.

[0051] Step S202: According to the first performance indicator and the first sub-rule, a server is selected from the server cluster to obtain a sub-cluster. The first sub-rule is used to select a server with high request processing efficiency from the server cluster. Taking the first performance indicator as the number of call failures as an example, the fewer the number of call failures, the stronger the server's processing capability for requests, so the first sub-rule can be the first M servers with the fewest call failures in the server cluster.

[0052] For another example, when the first performance indicator is throughput, the greater the throughput, the stronger the server's processing capability for requests, so the first sub-rule can be the first M servers with the largest throughput in the server cluster. Wherein, M is a custom integer, or the product of the total number of servers in the server cluster and the set first ratio. The first ratio can be custom set, such as 8%, 10%, 15%, etc.

[0053] Specifically, in this step, the servers in the server cluster may be sorted in order of the number of call failures from small to large (or the throughput from large to small), and then the servers ranked in the top 10% are selected and added to the sub-cluster.

[0054] Step S203: Determine whether the proportion of servers in the sub-cluster that meet the set reference value of the first performance indicator is greater than or equal to the set ratio threshold. If the proportion is less than the ratio threshold, execute step S204; if the proportion is greater than or equal to the ratio threshold, execute step S205. Both the reference value and the ratio threshold can be set according to requirements.

[0055] Taking the first performance indicator as the number of call failures as an example, the benchmark value can be set to 0, 1, etc., and the ratio threshold can be set to 90%, 95%, etc. This step first calculates the proportion of servers in the sub-cluster that meet the benchmark value for the first performance indicator, that is, calculates the proportion of servers in the sub-cluster whose number of call failures is 0 (or less than or equal to 1), and then compares the proportion with the ratio threshold.

[0056] Step S204: taking the sub-cluster as the first server subset, and executing step S206.

[0057] Step S205: According to the third performance indicator and the second sub-rule, a server is selected from the sub-cluster to obtain a first server subset, and step S206 is executed. The third performance indicator represents the server load, such as the number of concurrent requests and the number of user requests. The number of concurrent requests refers to the number of requests that the server can process at the same time; the number of user requests refers to the number of requests received by the server per unit time.

[0058] The second sub-rule is used to select a server with a low current load from the sub-cluster. Taking the third performance indicator as the number of concurrent requests as an example, the fewer the number of concurrent requests, the smaller the load of the server, so the second sub-rule can be the servers with the first N concurrent requests in the server cluster. Similarly, the smaller the number of user requests, the smaller the load of the server. Therefore, the second sub-rule can also be the servers with the first N user requests in the server cluster. Among them, N is a custom integer, or the product of the total number of servers in the sub-cluster and the set second ratio. The second ratio can be customized, such as 8%, 10%, 15%, etc. When the number of servers in a server cluster itself is relatively small, the proportion of servers with a call failure number of 0 is large. At this time, the resistance capacity of the server cluster (that is, the ability of the server cluster to withstand traffic and concurrent requests) will not be perceived, so it is necessary to convert the detection object, that is, from detecting the number of call failures to detecting the minimum number of concurrent requests, and then it can be clearly known which servers in the server cluster currently withstand the smallest number of concurrent requests, and network requests can be sent to these servers later.

[0059] Still taking the first performance indicator as the number of call failures and the third performance indicator as the number of concurrent requests as an example, if the proportion of servers with 0 call failures in the sub-cluster is greater than or equal to the proportion threshold, further servers are selected from the sub-cluster to obtain the first server subset. Figure 3 Description.

[0060] Step S206: Obtain the second performance indicator of the servers in the first server subset. Although the above steps obtain servers with a small number of concurrent requests, the performance of these servers is unknown, so it is necessary to use the second performance indicator for evaluation. In an embodiment, the TP999 of the most recent 100 calls to the servers in the first server subset can be obtained, and the TP999 can be obtained by averaging the built-in TP record container of each server. Assume that a server is requested 1000 times in 1 second, and arrange the time consumption of these requests in ascending order, where the average time consumption of the 990th to 999th requests is the value of TP999.

[0061] Through TP999, you can easily know which servers with a small number of concurrent requests have stronger processing capabilities, which have weaker processing capabilities, and which servers have poor current network conditions. This allows you to prioritize more network requests for servers with a small number of concurrent requests and low TP999 latency when allocating network requests later; prioritize fewer network requests for servers with a small number of concurrent requests but low TP999 latency.

[0062] Step S207: Determine whether the second performance indicator of the server in the first server subset is greater than or equal to the set indicator threshold. If the second performance indicator is greater than or equal to the indicator threshold, execute step S208; if the second performance indicator is less than the indicator threshold, execute step S209.

[0063] Step S208: Filter servers whose second performance index is greater than or equal to the index threshold from the first server subset. When TP999 is greater than or equal to the index threshold (timeout), for example, timeout is 1 second, although the number of concurrent requests of the server is small, the processing capacity of the server itself is already very poor, so the servers whose TP999 is greater than or equal to timeout in the first server subset need to be eliminated.

[0064] Step S209: The servers retained in the first server subset are used as the second server subset.

[0065] Step S210: Calculate the indicator weighted value of the server in the second server subset according to the weight information and the second performance indicator configured for the server. In the embodiment, the servers in the second server subset can be sorted in the order of TP999 from small to large, and it is determined whether each server is configured with weight information, and the indicator weighted value is calculated according to the determination result. The specific implementation of this step is described in the subsequent Figure 4 Description.

[0066] Step S211: Send the network request to the server corresponding to the minimum indicator weight value. In the above steps, the server is first filtered according to the first performance indicator (i.e., step S201-step S202), then filtered according to the third performance indicator (i.e., step S205), and finally filtered according to the second performance indicator (i.e., step S206-step S211), and the server with the best performance can be dynamically obtained.

[0067] This embodiment uses the number of call failures to evaluate the server's request processing capability, and filters servers in combination with server load conditions, so as to preferentially allocate network requests to servers with strong processing capabilities and low loads, thereby better evaluating cluster performance and making rational use of resources. At the same time, when the number of server call failures is small, servers are filtered according to the minimum number of concurrent requests, so that network requests can be preferentially allocated to servers with a small number of concurrent requests in the future.

[0068] Figure 3 FIG. 1 is a flowchart of selecting a server from a sub-cluster in an embodiment of the present invention. Figure 3 As shown, the implementation process of selecting a server from a sub-cluster (i.e., step S205) in the embodiment of the present invention mainly includes the following steps:

[0069] Step S301: Obtain the number of concurrent requests of the server in the sub-cluster. When obtaining the number of concurrent requests, the number of concurrent requests using calls within a period of time can be obtained. For example, the sampling values ​​of the 1st second sampling, the 10th second sampling, the 45th second sampling, and the 60th second sampling within 1 minute (this value is only an example, and any value that can accurately or relatively accurately reflect the server load situation can be replaced), and the sampling values ​​are summed to obtain the number of concurrent requests.

[0070] Step S302: sort the servers in the sub-cluster in order of the number of concurrent requests from small to large.

[0071] Step S303: adding servers whose concurrent request numbers satisfy the second sub-rule to the first server subset. In the embodiment, servers ranked in the top 10% are selected and added to the first server subset.

[0072] Figure 4 : is a flowchart of calculating the weighted value of an indicator in an embodiment of the present invention. Figure 4As shown, the implementation process of calculating the weighted index value (i.e., step S210) of the embodiment of the present invention mainly includes the following steps:

[0073] Step S401: determine whether the server in the second server subset is configured with weight information, if not, execute step S402; if configured with weight information, execute step S403. The weight information of the server can be customized.

[0074] Step S402: The second performance index is used as the index weighted value of the server, and the process ends. If the weight information is not set, it means that all servers have the same weight, and the second performance index of the server is the index weighted value of the server.

[0075] Step S403: Calculate the multiplication result of the second performance index of the server and the corresponding weight information, use the multiplication result as the index weight value of the server, and end this process. If weight information is set, calculate the index weight value according to the weight size.

[0076] For example, there are two servers in the second server subset, ProviderA and ProviderB, ProviderA's weight is 3, TP999 is 600ms; ProviderB's weight is 5, TP999 is 400ms. Then ProviderA's weighted index value = 3 × 600 = 1800ms, ProviderB's weighted index value = 5 × 400 = 2000ms. Since 2000>1800, ProviderA will be selected to allocate network requests first.

[0077] In a preferred embodiment, step S102 and step S103 may be executed cyclically. Due to the real-time detection of TP999, the load balancing of this embodiment has strong adaptability, and there will not be a situation where a server cluster itself has a small number of concurrent requests, but due to network jitter, the TP999 delay is high, resulting in a large number of network requests being backlogged.

[0078] At the same time, due to the real-time detection of TP999, the servers with high load in the server cluster are gradually assigned fewer network requests, so that their response capabilities are gradually restored; while the servers with low load in the server cluster are gradually assigned more network requests, making their resource utilization more reasonable. Ultimately, the CPU utilization of the entire server cluster is more balanced, the overall OPS response is gradually increasing, and the adaptability to network emergencies is also enhanced.

[0079] Figure 5The figure is a comparison of the effects before and after using the load balancing method of the embodiment of the present invention. Among them, (a) is a CPU usage curve of a server in a server cluster obtained by using a traditional load balancing method; (b) is a CPU usage curve of a server in the same server cluster obtained by using the load balancing method of the embodiment of the present invention.

[0080] Depend on Figure 5 It can be seen that in the traditional load balancing method, the CPU usage of the servers in the server cluster fluctuates between 30% and 70%, and the CPU usage cannot be well balanced. The load balancing method of the present invention, by taking into account the number of call failures, the number of concurrent requests, TP999 and weight, can perceive the overall processing capacity of the server, and the overall performance evaluation of the cluster is more comprehensive and balanced, thereby making the overall average CPU usage of the servers in the server cluster at 40%, which is more balanced than the traditional method.

[0081] Figure 6 FIG. 1 is a schematic diagram of the main modules of the load balancing device according to an embodiment of the present invention. Figure 6 As shown, the load balancing device 600 of the embodiment of the present invention mainly includes:

[0082] The first selection module 601 is used to obtain a first performance indicator of a server in the server cluster, and select a server from the server cluster according to the first performance indicator and a set screening rule to obtain a first server subset. The first performance indicator represents the request processing efficiency, such as the number of call failures, throughput, average throughput, etc.

[0083] The servers in the server cluster are traversed, and the first performance index of the servers is detected to obtain the request processing efficiency of each server. Then, a number of servers are selected from the server cluster according to the first performance index and the screening rule, and the selected servers constitute the first server subset. The screening rule is used to select servers with high request processing efficiency or low current load from the server cluster.

[0084] The second selection module 602 is used to obtain a second performance indicator of the server in the first server subset, and select a server from the first server subset according to the second performance indicator and a set indicator threshold to obtain a second server subset. The second performance indicator represents the request processing delay, such as a TP indicator, an average response time, etc.

[0085] In this module, the second performance index of the servers in the first server subset is first detected to obtain the request processing delay of each server. Then, based on the second performance index and the index threshold, the servers whose second performance index is less than or equal to the index threshold are selected from the first server subset, and the selected servers constitute the second server subset. The index threshold can be customized according to the needs.

[0086] The request sending module 603 is used to calculate the index weighted values ​​of the servers in the second server subset according to the weight information configured for the server and the second performance index, and send the network request to the server corresponding to the minimum index weighted value. This module determines whether the servers in the second server subset are configured with weight information. If the weight information is not configured, it means that the contributions of each server to the cluster are the same, and the network request is preferentially sent to the server corresponding to the minimum second performance index.

[0087] If weight information is configured, it means that the contribution of each server to the cluster varies. In this case, the weighted value of the indicator is calculated according to the weight, and the network request is sent to the server with the smallest weighted value of the indicator first.

[0088] In addition, the load balancing device 600 of the embodiment of the present invention may also include: a loop processing module ( Figure 6 ), which module is used to cyclically execute the processing of the second selection module 602 and the request sending module 603, so that the network requests sent to the servers with high load in the server cluster gradually decrease, and the network requests sent to the servers with low load in the server cluster gradually increase.

[0089] From the above description, it can be seen that by obtaining the first performance indicator and the second performance indicator to perceive the server performance and the real-time network status, and then according to the server performance and the real-time network status, the server is screened, and the number of allocated network requests is dynamically adjusted to avoid request backlogs. It has strong adaptability to network conditions and more rational resource utilization.

[0090] Figure 7 An exemplary system architecture 700 is shown to which the load balancing method or load balancing device according to the embodiment of the present invention can be applied.

[0091] like Figure 7As shown, the system architecture 700 may include terminal devices 701, 702, 703, networks 704, 706, a load balancing server 705, and servers 707, 708 belonging to the same server cluster. The network 704 is used to provide a medium for a communication link between the terminal devices 701, 702, 703 and the load balancing server 705. The network 706 is used to provide a medium for a communication link between the load balancing server 705 and the servers 707, 708. The networks 704, 706 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0092] Users can use terminal devices 701, 702, 703 to interact with load balancing server 705 through network 704 to receive or send messages, etc. Load balancing server 705 interacts with servers 707, 708 belonging to the same server cluster through network 706 to receive or send messages. Various communication client applications can be installed on terminal devices 701, 702.

[0093] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0094] The load balancing server 705 may be a server with functions such as receiving messages, forwarding messages, and implementing load balancing, for example, a background management server that processes messages sent by administrators using terminal devices 701, 702, and 703. The background management server may forward the message to one of the servers 707 and 708 according to a load balancing method.

[0095] It should be noted that the load balancing method provided in the embodiment of the present application is generally executed by the load balancing server 705 , and accordingly, the load balancing device is generally arranged in the load balancing server 705 .

[0096] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0097] According to an embodiment of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0098] The electronic device of the present invention comprises: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement a load balancing method of an embodiment of the present invention.

[0099] The computer-readable medium of the present invention stores a computer program, and when the program is executed by a processor, a load balancing method according to an embodiment of the present invention is implemented.

[0100] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 suitable for implementing an electronic device of an embodiment of the present invention. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0101] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the computer system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0102] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0103] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the main step diagram. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0104] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0105] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0106] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor, for example, may be described as: a processor includes a first selection module, a second selection module and a request sending module. The names of these modules do not constitute a limitation on the modules themselves in certain cases. For example, the first selection module may also be described as "obtaining a first performance indicator of a server in a server cluster, selecting a server from the server cluster according to the first performance indicator and the set screening rules, and obtaining a module of a first server subset".

[0107] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining a first performance indicator of a server in a server cluster, selecting a server from the server cluster according to the first performance indicator and a set screening rule, and obtaining a first server subset; wherein the first performance indicator represents the request processing efficiency; obtaining a second performance indicator of a server in the first server subset, selecting a server from the first server subset according to the second performance indicator and a set indicator threshold, and obtaining a second server subset; wherein the second performance indicator represents the request processing delay; calculating the indicator weighted value of the server in the second server subset according to the weight information configured for the server and the second performance indicator, and sending the network request to the server corresponding to the minimum indicator weighted value.

[0108] According to the technical solution of the embodiment of the present invention, the server performance and the real-time network status are perceived by acquiring the first performance indicator and the second performance indicator, and then the servers are screened according to the server performance and the real-time network status, and the number of allocated network requests is dynamically adjusted to avoid request backlogs. The system has strong adaptability to network conditions and more rational resource utilization.

[0109] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A load balancing method, characterized in that: include: Obtaining a first performance indicator of a server in a server cluster, selecting a server from the server cluster according to the first performance indicator and a set screening rule to obtain a first server subset, including: selecting a server from the server cluster according to the first performance indicator and a first sub-rule to obtain a sub-cluster; calculating the proportion of servers in the sub-cluster that meet a set benchmark value for the first performance indicator, and comparing the proportion with a set ratio threshold; according to the comparison result, taking the sub-cluster as the first server subset, or according to the comparison result, using a third performance indicator and a second sub-rule to select a server from the sub-cluster to obtain the first server subset; wherein the first performance indicator represents request processing efficiency; the screening rule includes a first sub-rule and a second sub-rule; Obtaining a second performance indicator of the servers in the first server subset, and selecting a server from the first server subset according to the second performance indicator and a set indicator threshold to obtain a second server subset; wherein the second performance indicator represents a request processing delay; According to the weight information configured for the server and the second performance indicator, the indicator weighted values ​​of the servers in the second server subset are calculated, and the network request is sent to the server corresponding to the minimum indicator weighted value.

2. The method according to claim 1, characterized in that The second performance indicator is a TP indicator or an average response time; According to the second performance indicator and the set indicator threshold, a server is selected from the first server subset to obtain a second server subset, including: Determining whether a second performance indicator of a server in the first server subset is greater than or equal to a set indicator threshold; The servers whose second performance index is greater than or equal to the index threshold are filtered from the first server subset to obtain the second server subset.

3. The method according to claim 1 or 2, characterized in that: Calculating the index weighted values ​​of the servers in the second server subset according to the weight information configured for the server and the second performance index includes: Determining whether the servers in the second server subset are configured with weight information; If the server is not configured with the weight information, using the second performance indicator as the indicator weighted value of the server; If the server is configured with the weight information, the multiplication result of the weight information and the second performance indicator is used as the indicator weighted value of the server.

4. The method according to claim 1, characterized in that: The first performance indicator is the number of call failures, throughput or average throughput; The third performance indicator represents the server load condition.

5. The method according to claim 4, characterized in that According to the comparison result, the sub-cluster is used as the first server subset, or according to the third performance indicator and the second sub-rule, a server is selected from the sub-cluster to obtain the first server subset, including: If the proportion is less than the ratio threshold, taking the sub-cluster as the first server subset; If the proportion is greater than or equal to the ratio threshold, a third performance indicator of the servers in the subcluster is obtained, and a server is selected from the subcluster according to the third performance indicator and the second sub-rule to obtain the first server subset.

6. The method according to claim 4 or 5, characterized in that: The third performance indicator is the number of concurrent requests or the number of user requests; The first sub-rule is: the first M servers with high request processing efficiency in the server cluster; wherein M is a user-defined integer, or the product of the total number of servers in the server cluster and the set first ratio; The second sub-rule is: the first N servers with low load in the sub-cluster; wherein N is a user-defined integer, or the product of the total number of servers in the sub-cluster and the set second ratio.

7. The method according to any one of claims 1-2, 4-5, characterized in that: The method further comprises: The obtaining and selecting step and the calculating and sending step are executed cyclically, so that the network requests sent to the servers with high load in the server cluster gradually decrease, and the network requests sent to the servers with low load in the server cluster gradually increase; Among them, the acquisition and selection step is a step of acquiring the second performance indicator and selecting a server from the first server subset; the calculation and sending step is a step of calculating the indicator weighted value and sending the network request to the server corresponding to the minimum indicator weighted value.

8. A load balancing device, characterized in that: include: The first selection module is used to obtain a first performance indicator of a server in a server cluster, select a server from the server cluster according to the first performance indicator and a set screening rule, and obtain a first server subset, including: selecting a server from the server cluster according to the first performance indicator and a first sub-rule to obtain a sub-cluster; calculating the proportion of servers in the sub-cluster that meet a set benchmark value for the first performance indicator, and comparing the proportion with a set ratio threshold; according to the comparison result, taking the sub-cluster as the first server subset, or according to the comparison result, using a third performance indicator and a second sub-rule to select a server from the sub-cluster to obtain the first server subset; wherein the first performance indicator represents request processing efficiency; the screening rule includes a first sub-rule and a second sub-rule; A second selection module is used to obtain a second performance indicator of the servers in the first server subset, and select servers from the first server subset according to the second performance indicator and a set indicator threshold to obtain a second server subset; wherein the second performance indicator represents a request processing delay; The request sending module is used to calculate the indicator weighted values ​​of the servers in the second server subset according to the weight information configured for the server and the second performance indicator, and send the network request to the server corresponding to the minimum indicator weighted value.

9. The device according to claim 8, characterized in that The second performance indicator is a TP indicator or an average response time; and the second selection module is further used for: Determining whether a second performance indicator of a server in the first server subset is greater than or equal to a set indicator threshold; and The servers whose second performance index is greater than or equal to the index threshold are filtered from the first server subset to obtain the second server subset.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images