A load balancing based request distribution method and apparatus

Abstract

The specification provides a load balancing-based request distribution method and device, and relates to the technical field of load balancing. The method comprises the following steps: monitoring the connection quantity of a plurality of service instances, the service instances corresponding to weights; determining whether the plurality of service instances satisfy a balance condition according to the connection quantity, the balance condition comprising: the proportion of the connection quantity between the service instances being equal to the proportion of the weights between the service instances; if the balance condition is not satisfied, selecting an abnormal service instance from the plurality of service instances; adjusting the weight of the abnormal service instance, so as to distribute connection requests to the abnormal service instance according to the adjusted weight, and make the connection quantity of the plurality of service instances satisfy the balance condition. The above method can avoid the problem of load inclination caused by uneven request distribution.

Description

Technical Field

[0001] This specification relates to the field of load balancing technology, and in particular to a request allocation method and apparatus based on load balancing. Background Technology

[0002] With the increasing number of internet applications and users, server load is also growing. Load balancing technology is one of the effective ways to solve the problem of excessive server load. Load balancing technology typically combines many servers into a server pool and routes requests to one of the servers. This can improve system availability, scalability, and performance. However, existing load balancing methods cannot avoid the problem of load skew among service instances under sudden surges while achieving low performance overhead.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This specification provides a load balancing-based request allocation method and apparatus to solve the problem that existing technologies cannot avoid load skew between service instances under sudden scenarios while achieving low performance overhead.

[0005] On the one hand, embodiments of this specification provide a request allocation method based on load balancing, including:

[0006] Monitor the number of connections to multiple service instances, each with a corresponding weight;

[0007] Based on the number of connections, it is determined whether the multiple service instances meet the balance condition, which includes: the ratio of the number of connections among the service instances is equal to the ratio of the weights among the service instances.

[0008] If the balance condition is not met, select an abnormal service instance from the plurality of service instances;

[0009] The weights of the abnormal service instances are adjusted so that connection requests are allocated to the abnormal service instances according to the adjusted weights, thereby ensuring that the number of connections for the multiple service instances meets the balance condition.

[0010] Further, determining whether the multiple service instances meet the load balancing condition based on the number of connections includes:

[0011] Based on the number of connections, determine the ratio of the number of connections among the multiple services; determine whether the ratio of the number of connections among the multiple service instances is equal to the ratio of the weights among the multiple service instances.

[0012] Furthermore, the failure to meet the equilibrium condition includes: the proportion of the number of connections being greater than the proportion of the weight and / or the proportion of the number of connections being less than the proportion of the weight.

[0013] Furthermore, adjusting the weight of the abnormal service instance includes:

[0014] If the proportion of the number of connections is greater than the proportion of the weight, then the weight of the abnormal service instance is reduced.

[0015] If the proportion of the number of connections is less than the proportion of the weight, then the weight of the abnormal service instance is increased.

[0016] Furthermore, the method also includes:

[0017] The abnormal service instance is assigned a corresponding connection request based on the reduced weight;

[0018] Monitor the number of connections received by the abnormal service instance after receiving a connection request;

[0019] Based on the number of connections, determine whether the multiple service instances meet the load balancing condition;

[0020] If not, the weight of the abnormal service instance is increased, and connection requests are allocated to the abnormal service instance according to the increased weight, so that the number of connections of the multiple service instances meets the balance condition.

[0021] Furthermore, the method also includes:

[0022] Obtain performance data of the servers hosting multiple service instances; the performance data includes memory utilization and processor utilization.

[0023] The weights of multiple service instances are determined based on the memory utilization and processor utilization.

[0024] Furthermore, the method also includes:

[0025] Receive a service call request, the service call request being used to request a service call;

[0026] Based on the adjusted weights, select a service instance from the plurality of service instances;

[0027] The service call request is assigned to the selected service instance so that the service call request establishes a connection with the selected service.

[0028] On the other hand, embodiments of this specification also provide a request allocation device based on load balancing, including:

[0029] The monitoring module is used to monitor the number of connections to multiple service instances, each of which has a corresponding weight.

[0030] The determination module is used to determine whether the multiple service instances meet the balance condition based on the number of connections. The balance condition includes: the ratio of the number of connections among service instances is equal to the ratio of the weights among service instances.

[0031] The selection module is used to select an abnormal service instance from the plurality of service instances if the balance condition is not met.

[0032] The allocation adjustment module is used to adjust the weight of the abnormal service instance so as to allocate connection requests to the abnormal service instance according to the adjusted weight, so that the number of connections of the multiple service instances meets the balance condition.

[0033] Furthermore, this application also provides an electronic device, including a processor and a memory for storing processor-executable instructions. When the processor executes the instructions, it performs the following: monitoring the number of connections to multiple service instances, each service instance having a corresponding weight; determining whether the multiple service instances meet a balance condition based on the number of connections, the balance condition including: the ratio of the number of connections between service instances equals the ratio of the weights between service instances; if the balance condition is not met, selecting an abnormal service instance from the multiple service instances; adjusting the weight of the abnormal service instance so that connection requests are allocated to the abnormal service instance according to the adjusted weight, so that the number of connections to the multiple service instances meets the balance condition.

[0034] Furthermore, this application also provides a computer-readable storage medium storing computer instructions thereon. When the computer-readable storage medium executes the instructions, it performs the following: monitoring the number of connections to multiple service instances, each service instance having a corresponding weight; determining whether the multiple service instances meet a balance condition based on the number of connections, the balance condition including: the ratio of the number of connections between service instances equals the ratio of the weights between service instances; if the balance condition is not met, selecting an abnormal service instance from the multiple service instances; adjusting the weight of the abnormal service instance so as to allocate connection requests to the abnormal service instance according to the adjusted weight, so that the number of connections to the multiple service instances meets the balance condition.

[0035] This specification provides a request allocation method and apparatus based on load balancing. First, it monitors the connection counts of multiple service instances, each with a corresponding weight. Second, based on the connection counts, it determines whether the multiple service instances meet load balancing conditions. These conditions include: the ratio of connection counts among service instances equals the ratio of weights among service instances. If the load balancing conditions are not met, an abnormal service instance is selected from the multiple service instances. Finally, the weights of the abnormal service instances are adjusted so that connection requests are allocated to them according to the adjusted weights, ensuring that the connection counts of the multiple service instances meet the load balancing conditions. In this embodiment, by monitoring the connection counts of multiple service instances, it is possible to promptly determine whether the multiple service instances meet the load balancing conditions, thereby enabling the timely filtering of abnormal service instances. By adjusting the weights of abnormal service instances, it is possible to promptly address load skew issues among service instances in sudden scenarios. Attached Figure Description

[0036] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a request allocation method based on load balancing provided in the embodiments of this specification.

[0038] Figure 2 This is a schematic diagram illustrating an embodiment of a load-balancing-based request allocation method provided in this specification, presented in a scenario example.

[0039] Figure 3 This is a schematic diagram of the structural composition of a request allocation device based on load balancing provided in the embodiments of this specification;

[0040] Figure 4 This is a schematic diagram of the structural composition of the computer device provided in the embodiments of this specification. Detailed Implementation

[0041] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0042] With the increasing number of internet applications and users, server load is also growing. Load balancing technology is one of the effective ways to solve the problem of excessive server load. Load balancing typically combines many servers into a server pool and routes requests to one of those servers. This improves system availability, scalability, and performance. Common load balancing algorithms include:

[0043] 1. Random or Weighted Random Algorithm: Randomly selects a service instance to allocate tasks. This ensures the distribution of requests, thus achieving load balancing. Weighted randomness adjusts the probability of randomness by configuring a weight for each service instance.

[0044] 2. Round-robin or weighted round-robin algorithms: These algorithms distribute requests to various service instances in turn. Weighted round-robin, in addition to round-robin, distributes requests according to given weights.

[0045] 3. Least Connections Algorithm: Distribute requests to the node with the fewest connections in the service instance.

[0046] 4. Hash method: A value is calculated using a hash function based on the request content. This value is then moduloed by the size of the server list. The result is the sequence number of the service instance that the caller wants to access. This ensures that the same caller always requests the same server. It is mainly used for stateful service instances.

[0047] However, existing load balancers have the following problems:

[0048] The drawbacks of random or weighted random algorithms, and round-robin or weighted round-robin algorithms are as follows: While they can distribute requests evenly across different service instances and differentiate backend service instances with varying processing capabilities by adding weights (e.g., assigning higher weights to higher-performing service instances), they can lead to load skew across service instances in certain unexpected scenarios. For example, network fluctuations or physical machine failures may cause some service instances to process requests slower than the remaining instances. In this case, although requests are evenly forwarded to all service instances, there may still be a backlog of requests on slower service instances while normal instances are idle.

[0049] For the least connections algorithm: Although it can effectively send more requests to normal service instances in the above-mentioned sudden situations, it consumes more performance compared to random or weighted random algorithms, round-robin or weighted round-robin algorithms (e.g., it needs to traverse all service instances to find the one with the fewest connections each time a request is allocated).

[0050] To address the aforementioned problems with existing methods, this specification introduces a request allocation method and apparatus based on load balancing. It considers combining the advantages of random or round-robin algorithms with the least connection algorithm to achieve low performance overhead while avoiding the problem of load imbalance among service instances under sudden scenarios.

[0051] Based on the above approach, this specification proposes a request allocation method based on load balancing. First, it monitors the connection counts of multiple service instances, each with a corresponding weight. Second, based on the connection counts, it determines whether the multiple service instances meet a load balancing condition. This condition includes that the ratio of connection counts among service instances equals the ratio of weights among service instances. If the load balancing condition is not met, an abnormal service instance is selected from the multiple service instances. Finally, the weights of the abnormal service instances are adjusted so that connection requests are allocated to them according to the adjusted weights, ensuring that the connection counts of the multiple service instances meet the load balancing condition. (See also...) Figure 1 As shown in the illustration, this specification provides a request allocation method based on load balancing. In specific implementation, this method may include the following:

[0052] S101: Monitor the number of connections to multiple service instances, each with a corresponding weight.

[0053] In some embodiments, the multiple service instances described above may correspond to weights and connection counts, where the weights and connection counts can be initial weights and initial connection counts. The initial weights can be obtained in the following manner:

[0054] First, performance data of the servers hosting multiple service instances can be obtained. This performance data may include memory utilization and processor utilization. Then, based on the memory utilization and processor utilization, the weights (initial weights) corresponding to the multiple service instances can be determined. The performance data may also include: disk utilization, disk I / O frequency, processor (CPU) capacity, memory capacity, etc., but this specification does not specifically limit these.

[0055] You can also perform a weighted summation of memory utilization and processor utilization to determine the load metrics of the servers hosting multiple service instances. Then, you can perform a weighted summation of processor (CPU) capacity and memory capacity to determine the performance metrics of the servers hosting multiple service instances. Finally, you can determine the initial weights mentioned above based on the ratio of performance metrics to load metrics.

[0056] In this context, a higher weight indicates a stronger processing capability for the service instance, while a lower weight indicates a weaker processing capability. A higher number of connections means a service instance can handle fewer requests, while a lower number of connections means a service instance can handle more requests.

[0057] In some embodiments, connection requests can be allocated to each service instance based on the weights corresponding to the multiple service instances, and then the number of connections received by the multiple service instances after receiving the connection requests can be monitored. That is, connection requests can be evenly distributed to each service instance according to the initial weights, and then the number of connections received by each service instance after receiving the requests can be monitored. It should be noted that the initial number of connections of each service instance can change after receiving and processing the connection requests, and the number of connections of the multiple service instances monitored can be the number of connections sent after the change. By evenly distributing requests first and then monitoring the number of connections, performance overhead can be reduced, and load skew caused by network fluctuations, physical machine failures, etc. can also be effectively avoided.

[0058] S102: Based on the number of connections, determine whether the multiple service instances meet the balance condition, wherein the balance condition includes: the ratio of the number of connections among service instances is equal to the ratio of the weights among service instances.

[0059] In some embodiments, determining whether the plurality of service instances meet the balance condition based on the number of connections may include: determining the ratio of the number of connections among the plurality of services based on the number of connections; comparing the ratio of the number of connections with the ratio of the weights to determine whether the ratio of the number of connections among the plurality of service instances is equal to the ratio of the weights among the plurality of service instances. For example: service instance A has a weight of 1, service instance B has a weight of 1, and service instance C has a weight of 3. If the number of connections for service instance A is 2, the number of connections for service instance B is 2, and the number of connections for service instance C is 3, then the ratio of the number of connections for service instance A, service instance B, and service instance C is 2:2:3, and the ratio of the weights is 1:1:3. Since the ratio of the number of connections is not equal to the ratio of the weights, service instance A, service instance B, and service instance C are considered not to meet the balance condition.

[0060] By monitoring the number of connections to service instances in real time, and determining whether multiple service instances meet the load balancing conditions based on the number of connections, it is possible to promptly detect issues such as load skew caused by network fluctuations.

[0061] S103: If the balance condition is not met, select an abnormal service instance from the multiple service instances.

[0062] In some embodiments, the failure to meet the load balancing condition may include: the proportion of the number of connections being greater than the proportion of the weight and / or the proportion of the number of connections being less than the proportion of the weight. There may be one or more abnormal service instances. An abnormal service instance may be a service instance whose number of connections increases or decreases due to network fluctuations, and the proportion of the increased or decreased connection number is inconsistent with the proportion of the weight. By filtering abnormal service instances, a foundation can be laid for dynamically adjusting the weights of abnormal service instances subsequently, preventing load skew among service instances.

[0063] In some embodiments, determining whether the multiple service instances meet the load balancing condition based on the number of connections may further include: if so, continuing to allocate connection requests according to the weights corresponding to the service instances, monitoring the number of connections received by the multiple service instances after receiving connection requests, and again determining whether the multiple service instances meet the load balancing condition based on the number of connections. By allocating connection requests based on the initial weights of the service instances when multiple service instances meet the load balancing condition, and promptly filtering out abnormal service instances when multiple service instances do not meet the load balancing condition, performance consumption can be reduced while avoiding load skew among the service instances.

[0064] S104: Adjust the weight of the abnormal service instance so that connection requests are allocated to the abnormal service instance according to the adjusted weight, so that the number of connections of the multiple service instances meets the balance condition.

[0065] In some embodiments, adjusting the weight of the abnormal service instance described above may include:

[0066] If the proportion of the number of connections is greater than the proportion of the weight, then the weight of the abnormal service instance is reduced.

[0067] If the proportion of the number of connections is less than the proportion of the weight, then the weight of the abnormal service instance is increased.

[0068] In some embodiments, connection requests may be allocated to the abnormal service instances according to the reduced weights; the number of connections of the abnormal service instances after receiving connection requests may be monitored; based on the number of connections, it may be determined whether the multiple service instances meet the balance condition; if not, the weights of the abnormal service instances may be increased so that connection requests are allocated to the abnormal service instances according to the increased weights, thereby ensuring that the number of connections of the multiple service instances meets the balance condition.

[0069] In some embodiments, sudden network fluctuations may cause a significant increase in the number of connections to one or more service instances, leading to a decrease in the processing performance of those service instances. This results in the proportion of connections to that service instance exceeding its initial weight. In such cases, the weight of the service instance needs to be reduced so that the service instance affected by the network fluctuation is allocated fewer requests, ensuring that the current request allocation to the service instance is compatible with its current processing capacity. After the number of allocated requests decreases, the processing capacity of the abnormal service instance may recover. The weight of the abnormal service instance can then be increased again, ensuring that the number of connections between the increased-weight abnormal service instance and other service instances meets the initial weight ratio. This guarantees that all service instances are functioning normally.

[0070] In some embodiments, taking a specific scenario as an example: Suppose there are three service instances A, B, and C. The processing capacity and network status of these three service instances are completely consistent, and the initial weight can be set to 1:1:1. During the stable operation phase, the number of connections for the three service instances A, B, and C are roughly equal, and the weights will not be adjusted. However, when the network of the node of service instance A suddenly fluctuates, the processing capacity of service instance A decreases, and requests begin to accumulate on service instance A. At this time, monitoring shows that service instance A does not meet the balance condition, such as: the number of connections of service instance A increases significantly, causing the proportion of connections of A to be greater than the proportion of the initial weight. At this time, the weight of service instance A can be reduced, for example, to 0.5:1:1, so that the number of requests forwarded to service instance A decreases until service instance A returns to normal. The number of connections of the node of service instance A will be significantly less than the 0.5:1:1 ratio. At this time, the weight of service instance A can be increased again until it is restored to the initial weight ratio of 1:1:1 (the proportion restored to the initial weight is the proportion when all service instances are normal).

[0071] In some embodiments, a service invocation request sent by a user can also be received, which can be used to request a service invocation. Then, a service instance is selected from the plurality of service instances according to the adjusted weights. Finally, the service invocation request is assigned to the selected service instance to establish a connection between the service invocation request and the selected service. By using adjusted weights to select a service instance from multiple service instances and then assigning the service invocation request to the selected service instance, the corresponding service can be invoked accurately and quickly, and a connection between the service request and the service can be established.

[0072] The above method will be described below with reference to a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustration of this application and does not constitute an improper limitation of this application.

[0073] Before implementation, firstly, the initial weights and initial connection counts of multiple service instances can be obtained, with the initial weights and initial connection counts being in equal proportions. Then, connection requests can be allocated to the multiple service instances based on their initial weights; service instances with higher initial weights can be allocated more requests, and service instances with lower initial weights can be allocated fewer requests. Finally, each service instance can process the allocated connection requests.

[0074] In practical implementation, a dynamic load balancer can monitor the connection count of multiple service instances in real time. The presence of abnormal service instances can be determined by comparing the connection count ratio with the initial weight ratio. If the connection count ratio does not equal the initial weight ratio, abnormal service instances are identified, and their initial weights are dynamically adjusted. Connection requests are then allocated to these abnormal service instances based on their dynamically adjusted weights until the connection count ratio of all service instances matches or equals the initial weight ratio. If the connection count ratio still equals the initial weight ratio, the initial weights are reassigned to the service instances, and the connection count of each service instance after receiving a connection request continues to be monitored. By continuously allocating connection requests and monitoring the connection count, the request allocation method can be adjusted promptly when network fluctuations occur. For example, when multiple service instances meet the load balancing condition (no network fluctuations or other abnormalities, no abnormal service instances), connection requests can be allocated based on the service instance weights. This ensures that each service instance receives a uniform distribution of requests, and the round-robin allocation reduces performance overhead. When multiple service instances do not meet the load balancing conditions (due to abnormal situations such as network fluctuations, resulting in abnormal service instances), abnormal service instances can be filtered out and corresponding requests can be allocated to them. This allows abnormal service instances to handle fewer requests when their processing performance is reduced due to network fluctuations. After a period of time, once the processing performance of the abnormal service instances has improved, they can then handle more requests.

[0075] In a specific scenario example, a load-balancing-based request allocation method provided in the embodiments of this specification can be applied to reduce performance consumption while avoiding load skew among service instances caused by uneven request distribution. See also... Figure 2 As shown, it may include:

[0076] Service Instance Pool: The service instance pool stores all current services and their corresponding service instances, such as Service 1 and Service 2, as well as service instances 1A, 1B, and 1C in Service 1 and service instances 2A, 2B, and 2C in Service 2. Service 1 and Service 2 are two distinct services, each with multiple corresponding service instances. A service instance is a concrete implementation of a service. At the deployment level, a service may consist of one or more service instances. Each service instance can fully handle all requests for that service.

[0077] Initially, each service instance can be configured with its own initial weight based on its actual processing capacity. For example, the initial weights configured for service instances 1A, 1B, and 1C are 1, 1, and 1, respectively, with current connection counts of 10, 10, and 10. The initial weights configured for service instances 2A, 2B, and 2C are 1, 0.5, and 2, respectively, with current connection counts of 10, 20, and 5. The ratio of the initial weights and the ratio of the current connection counts are the same or consistent.

[0078] Weighted Round Robin Selector: Obtains a service instance from the services requested by the service caller (the requester, which can be the user or client that initiated the request) based on weight, and forwards it to the backend service instance.

[0079] Dynamic load balancer: Periodically queries the connection count of each service instance in the service instance pool and dynamically adjusts the weights based on the current connection count to ensure that the ratio of requests handled by each service instance remains consistent with its initial weight. For example, service instances 1A, 1B, and 1C may have initial weights of 1:1:2, but actual connection counts of 2:1:3. In this case, 1A handles fewer requests and needs a higher weight, while 1C handles more requests and needs a lower weight. After adjusting the weights, the ratio of connections handled by each instance can be restored to the initial weight ratio of 1:1:2.

[0080] By using a weighted round-robin selector, requests can be evenly distributed across service instances. Directly allocating requests based on weighted round-robin reduces performance overhead and improves request allocation efficiency. By using a dynamic load balancer, anomalies in the service instance pool can be monitored in real time, and the weights of service instances can be adjusted promptly. This ensures that the allocated requests are compatible with the current processing capacity of the service instances, thus addressing unexpected scenarios such as network fluctuations or physical machine failures that could lead to uneven service instance loads.

[0081] Although this specification provides the following examples or appendices Figure 3The methods, steps, or apparatus structures shown may include more or fewer combined operational steps or module units based on conventional or non-inventive methods. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure shown in the embodiments or drawings of this specification. When the methods or module structures described are applied in actual devices, servers, or terminal products, they can be executed sequentially or in parallel according to the methods or module structures shown in the embodiments or drawings (e.g., in parallel processor or multi-threaded processing environments, or even distributed processing or server cluster implementation environments).

[0082] Based on the above-described load-balancing-based request allocation method, this specification also proposes an embodiment of a load-balancing-based request allocation device. For example... Figure 3 As shown, the device may specifically include the following modules:

[0083] Monitoring module 301 can be used to monitor the number of connections to multiple service instances, each service instance having a corresponding weight.

[0084] The determining module 302 can be used to determine whether the multiple service instances meet the balance condition based on the number of connections. The balance condition includes: the ratio of the number of connections among service instances is equal to the ratio of the weights among service instances.

[0085] The selection module 303 can be used to select an abnormal service instance from the plurality of service instances if the balancing condition is not met.

[0086] The adjustment allocation module 304 can be used to adjust the weight of the abnormal service instance so as to allocate connection requests to the abnormal service instance according to the adjusted weight, so that the number of connections of the multiple service instances meets the balance condition.

[0087] In some embodiments, the weights in the monitoring module 301 described above can be obtained as follows: obtain performance data of the servers where multiple service instances are located; the performance data includes memory utilization and processor utilization; determine the weights corresponding to the multiple service instances based on the memory utilization and processor utilization.

[0088] In some embodiments, the determining module 302 may also be used to determine the ratio of the number of connections among the plurality of services based on the number of connections; and to determine whether the ratio of the number of connections among the plurality of service instances is equal to the ratio of the weights among the plurality of service instances.

[0089] In some embodiments, the condition of not satisfying the balance in the selection module 303 may include: the proportion of the number of connections is greater than the proportion of the weight and / or the proportion of the number of connections is less than the proportion of the weight.

[0090] In some embodiments, the aforementioned adjustment allocation module 304 may be used to reduce the weight of the abnormal service instance when the proportion of the number of connections is greater than the proportion of the weight; and to increase the weight of the abnormal service instance when the proportion of the number of connections is less than the proportion of the weight.

[0091] In some embodiments, the adjustment and allocation module 304 may also be used to allocate corresponding connection requests to the abnormal service instance according to the reduced weight; monitor the number of connections of the abnormal service instance after receiving the connection request; determine whether the multiple service instances meet the balance condition according to the number of connections; if not, increase the weight of the abnormal service instance so as to allocate connection requests to the abnormal service instance according to the increased weight, so that the number of connections of the multiple service instances meets the balance condition.

[0092] In some embodiments, the adjustment and allocation module 304 may further be used to receive service call requests, which are used to request to call a service; select a service instance from the plurality of service instances according to the adjusted weights; and allocate the service call request to the selected service instance so that the service call request establishes a connection with the selected service.

[0093] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0094] As can be seen from the above, the load-balancing-based request allocation device provided in the embodiments of this specification can monitor the number of connections to multiple service instances and determine whether the service instances meet the load-balancing conditions based on the number of connections, thus enabling timely and accurate filtering of abnormal service instances. After filtering out abnormal service instances, by dynamically adjusting the initial weight of the abnormal service instances, the abnormal service instances can be allocated to handle fewer requests when performance is low, and then handle more requests after performance recovers, thereby avoiding load imbalance among multiple service instances. Furthermore, it can effectively address the problem of sudden performance degradation of one or more service instances caused by network anomalies or other situations.

[0095] This specification also provides an electronic device, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can perform the following steps according to the instructions: monitoring the connection counts of multiple service instances, each service instance having a corresponding weight; determining whether the multiple service instances meet a balance condition based on the connection counts, the balance condition including: the ratio of the number of connections between service instances equals the ratio of the weights between service instances; if the balance condition is not met, selecting an abnormal service instance from the multiple service instances; adjusting the weight of the abnormal service instance so that connection requests are allocated to the abnormal service instance according to the adjusted weight, thereby ensuring that the number of connections of the multiple service instances meets the balance condition.

[0096] To execute the above instructions more accurately, please refer to... Figure 4 As shown in the embodiments of this specification, another specific electronic device is also provided, wherein the electronic device includes a network communication port 401, a processor 402 and a memory 403, and the above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0097] Specifically, the network communication port 401 can be used to monitor the number of connections to multiple service instances, and each service instance has a corresponding weight.

[0098] The processor 402 can be specifically used to determine whether the plurality of service instances meet the balance condition based on the number of connections. The balance condition includes: the ratio of the number of connections among service instances is equal to the ratio of the weights among service instances; if the balance condition is not met, select an abnormal service instance from the plurality of service instances; adjust the weight of the abnormal service instance so as to allocate connection requests to the abnormal service instance according to the adjusted weight, so that the number of connections of the plurality of service instances meets the balance condition.

[0099] The memory 403 can be used to store the corresponding instruction program.

[0100] In this embodiment, the network communication port 401 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0101] In this embodiment, the processor 402 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0102] In this embodiment, the memory 403 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0103] This specification also provides a computer storage medium for a load balancing-based request allocation method. The computer storage medium stores computer program instructions that, when executed, implement the following: monitoring the connection counts of multiple service instances, each service instance having a corresponding weight; determining whether the multiple service instances meet a load balancing condition based on the connection counts, the load balancing condition including: the ratio of connection counts among service instances equals the ratio of weights among service instances; if the load balancing condition is not met, selecting an abnormal service instance from the multiple service instances; adjusting the weight of the abnormal service instance so that connection requests are allocated to the abnormal service instance according to the adjusted weight, thereby ensuring that the connection counts of the multiple service instances meet the load balancing condition.

[0104] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0105] This specification also provides a computer program product for a load balancing-based request allocation method. The computer program product includes a computer program that, when executed by a processor, performs the following: monitoring the number of connections to multiple service instances, each service instance having a corresponding weight; determining whether the multiple service instances meet a load balancing condition based on the number of connections, the load balancing condition including: the ratio of the number of connections between service instances equals the ratio of the weights between service instances; if the load balancing condition is not met, selecting an abnormal service instance from the multiple service instances; adjusting the weight of the abnormal service instance so that connection requests are allocated to the abnormal service instance according to the adjusted weight, thereby ensuring that the number of connections to the multiple service instances meets the load balancing condition.

[0106] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0107] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0108] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0109] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0110] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0111] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations of this specification are possible without departing from its spirit, and it is intended that the appended claims cover such variations without departing from the spirit of this specification.

Claims

1. A request allocation method based on load balancing, characterized in that, include: Based on the initial weights corresponding to multiple service instances obtained by weighted round-robin, connection requests are allocated to each service instance, and the number of connections of multiple service instances is monitored. The initial weights are determined based on the ratio of the performance indicators and load indicators of the servers where the multiple service instances are located. The performance indicators are determined by weighted summation of processor capacity and memory, and the load indicators are determined by weighted summation of memory utilization and processor utilization. Based on the number of connections, it is determined whether the multiple service instances meet the balance condition, which includes: the ratio of the number of connections among the service instances is equal to the ratio of the weights among the service instances. If the balance condition is not met, select an abnormal service instance from the plurality of service instances; The weights of the abnormal service instances are adjusted so that connection requests are allocated to the abnormal service instances according to the adjusted weights, so that the number of connections of the multiple service instances meets the balance condition. The adjustment of the weight of the abnormal service instance includes: If the proportion of the number of connections is greater than the proportion of the weight, then the weight of the abnormal service instance is reduced. The method further includes: Based on the reduced weights, corresponding connection requests are allocated to the abnormal service instances using a weighted round-robin method. Monitor the number of connections received by the abnormal service instance after receiving a connection request; Based on the number of connections, determine whether the multiple service instances meet the load balancing condition; If not, the weight of the abnormal service instance is increased, and connection requests are allocated to the abnormal service instance through weight round-robin based on the increased weight, so that the number of connections of the multiple service instances meets the balance condition. If so, then the connection requests will continue to be allocated according to the weight of the service instance through weight round-robin, and the number of connections of multiple service instances after receiving the connection request will be monitored, and the balance condition of multiple service instances will be determined again based on the number of connections. The method further includes: receiving a service call request sent by a user, the service call request being used to request a service call; selecting a service instance from the plurality of service instances according to the adjusted weights; and assigning the service call request to the selected service instance so that the service call request establishes a connection with the selected service.

2. The method according to claim 1, characterized in that, The step of determining whether the multiple service instances meet the load balancing condition based on the number of connections includes: Based on the number of connections, determine the ratio of the number of connections among the multiple services; determine whether the ratio of the number of connections among the multiple service instances is equal to the ratio of the weights among the multiple service instances.

3. The method according to claim 2, characterized in that, The conditions for not meeting the balance include: the proportion of the number of connections being greater than the proportion of the weight and / or the proportion of the number of connections being less than the proportion of the weight.

4. The method according to claim 3, characterized in that, The adjustment of the weight of the abnormal service instance also includes: If the proportion of the number of connections is less than the proportion of the weight, then the weight of the abnormal service instance is increased.

5. A request allocation device based on load balancing, characterized in that, include: The monitoring module is used to allocate connection requests to each service instance based on the initial weights corresponding to multiple service instances obtained by weighted round-robin, and then monitor the number of connections of multiple service instances. The initial weights are determined based on the ratio of the performance indicators and load indicators of the servers where the multiple service instances are located. The performance indicators are determined by weighted summation of processor capacity and memory, and the load indicators are determined by weighted summation of memory utilization and processor utilization. The determination module is used to determine whether the multiple service instances meet the balance condition based on the number of connections. The balance condition includes: the ratio of the number of connections among service instances is equal to the ratio of the weights among service instances. The selection module is used to select an abnormal service instance from the plurality of service instances if the balance condition is not met. The allocation adjustment module is used to adjust the weight of the abnormal service instance so as to allocate connection requests to the abnormal service instance according to the adjusted weight, so that the number of connections of the multiple service instances meets the balance condition. The adjustment of the weight of the abnormal service instance includes: If the proportion of the number of connections is greater than the proportion of the weight, then the weight of the abnormal service instance is reduced. The device further includes: Based on the reduced weights, corresponding connection requests are allocated to the abnormal service instances using a weighted round-robin method. Monitor the number of connections received by the abnormal service instance after receiving a connection request; Based on the number of connections, determine whether the multiple service instances meet the load balancing condition; If not, the weight of the abnormal service instance is increased, and connection requests are allocated to the abnormal service instance through weight round-robin based on the increased weight, so that the number of connections of the multiple service instances meets the balance condition. If so, then the connection requests will continue to be allocated according to the weight of the service instance through weight round-robin, and the number of connections of multiple service instances after receiving the connection request will be monitored, and the balance condition of multiple service instances will be determined again based on the number of connections. The apparatus further includes: receiving a service call request sent by a user, the service call request being used to request a service call; selecting a service instance from the plurality of service instances according to an adjusted weight; and assigning the service call request to the selected service instance so that the service call request establishes a connection with the selected service.

6. An electronic device, characterized in that, The method includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to implement the steps of the method according to any one of claims 1 to 4.

7. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 4.

8. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.

Images