Dynamic adaptive client load balancing method and system for microservices

Abstract

The present invention belongs to the field of server load balancing, and relates to a dynamic adaptive client load balancing method and system for microservices. The method comprises: a client node periodically sending a service request to each server node in a microservice cluster, so as to acquire load information of server nodes and cache the load information in Redis; when an actual load rate and a comprehensive load rate of the load information of each server node in the microservice cluster exceed corresponding threshold values, using a dynamic feedback weight load balancing algorithm to determine the server node which currently has the greatest weight to execute the request; and adjusting a traffic control strategy on the basis of load conditions of the server nodes, and once a traffic control condition threshold value is triggered, dynamically limiting the volume of incoming traffic according to a certain strategy, so as to reduce the load pressure on the nodes. In the present invention, client load balancing is used, so that the burden on a serving end is reduced; and loads are dynamically adjusted on the basis of actual load conditions of server nodes, so that the present invention has good practical significance and application value.

Description

A microservice dynamic adaptive client load balancing method and system Technical Field

[0001] The present invention belongs to the field of server load balancing and proposes a microservice dynamic adaptive client load balancing method and system. Background Art

[0002] With the advent of the "Internet Plus" era, microservices architectures have garnered increasing attention. Traditional monolithic services were typically deployed on a single server, making them difficult to scale and maintain. Therefore, microservices architectures have become a necessary alternative. However, clustering is often employed within microservices architectures to address high concurrency. This inevitably leads to some issues. For example, current load balancing algorithms can easily lead to uneven allocation of server resources, resulting in delayed responses to user requests and undermining the full effectiveness of clustering. Furthermore, when IoT terminals periodically transmit data, while each data request is relatively small, the transmission frequency is high, potentially leading to tens or even hundreds of thousands of requests simultaneously. This large number of requests and high concurrency complicates processing. Therefore, in server cluster systems, an effective load balancing algorithm is required to fully utilize the computing resources of each cluster node and improve system performance.

[0003] Load balancing has always been an important research topic, aiming to ensure that computing resources are distributed efficiently and fairly. Depending on the implementation method, existing load balancing can be divided into server-side load balancing as shown in Figure 1, and client-side load balancing as shown in Figure 2. Among them, client-side load balancing integrates the load balancer into the client application, through which it selects the appropriate server to send requests to reduce the pressure on the server. Therefore, client-side load balancing is widely used in practical applications. Existing client-side load balancing provides a variety of load balancing algorithms, including polling, weighted polling, and least connection algorithms, but these algorithms cannot adaptively adjust when the load changes, or only consider a single performance indicator, lacking a comprehensive evaluation of the load status of the server cluster and an accurate evaluation of the real-time load status of the server nodes in order to appropriately allocate request tasks.

[0004] Relevant scholars have proposed some dynamic load balancing algorithms to address the shortcomings of existing load balancing algorithms. However, these algorithms usually do not take into account the differences in node resource consumption due to different task types and the inability to effectively handle high pressure when the total amount of system concurrency is large. The results show that under high load conditions, these algorithms usually improve efficiency, but under low load conditions, their efficiency improvement is not obvious compared to traditional algorithms.

[0005] Summary of the Invention

[0006] In response to the above problems, considering that the strategy provided by the Ribbon client load balancer is more flexible, the present invention proposes a microservice dynamic adaptive client load balancing method and system. This method takes into account the actual load status of the server nodes in the microservice cluster and the differences in resource consumption of different request task types. In order to solve the problem of limited efficiency improvement of the existing dynamic load balancing algorithm under low load conditions, this method periodically collects the dynamic performance of the back-end server nodes, and selects the dynamic weight feedback load balancing algorithm or the smooth weighted polling load balancing algorithm suitable for the scenario as needed. The dynamic weight feedback load balancing algorithm proposed in the present invention combines the initial weight and the dynamic weight to determine the optimal weight of the server node, and continuously updates the old weight and weight coefficient when necessary. At the same time, under high load pressure, further business flow control measures are taken within the node to make full use of resources and achieve the goal of load balancing. The technical solution provided by the present invention specifically includes:

[0007] A first aspect of the present invention provides a microservice dynamic adaptive client load balancing method, which is oriented to periodic services of Internet of Things terminals and includes the following steps:

[0008] The client node regularly sends service requests to each server node in the microservice cluster to obtain performance parameter load information of the server node and caches the load information in Redis; the load information includes CPU utilization, memory utilization, network bandwidth utilization, and disk I / O utilization;

[0009] The client node makes a judgment based on the actual load rate and the actual load rate threshold of each server node in the microservice cluster, or / and the size of the comprehensive load rate and the comprehensive load rate threshold. If the load rate exceeds the corresponding threshold, the dynamic feedback weight load balancing algorithm is used to determine the server node with the largest current weight.

[0010] The client node sends a call request to the server node with the largest current weight in the microservice cluster, executes the request in the corresponding microservice execution module, and returns the execution result to the client node.

[0011] Furthermore, the present invention also includes adopting a flow control strategy to adjust the deviation between the time when each client node first sends a service request and the system start time, on the premise of minimizing the amount of service requests received by the server node within a certain period of time, if the CPU utilization and / or memory utilization of the server node periodically obtained by the client node exceeds the corresponding preset threshold.

[0012] A second aspect of the present invention provides a microservice dynamic adaptive client load balancing system for implementing the microservice dynamic adaptive client load balancing method described in the first aspect of the present invention. The system includes at least a client node, a server node, and a registration center; the client node includes a device / gateway, a management platform, a service gateway, and a load balancer; the server node includes a server load information collection module and a microservice execution module, and multiple server nodes constitute a microservice cluster; the client node and the server node are respectively registered in the registration center;

[0013] The client node initiates a request to each server node in the microservice cluster through the device / gateway and service gateway; the server node collects the load information of the server node through the server load information collection module, processes the collected load information through the dynamic feedback weight load balancing algorithm equipped with the load balancer, and determines the server node with the largest current weight; sends a call request to the server node with the largest current weight in the microservice cluster, executes the request in the corresponding microservice execution module, and returns the execution result to the client node.

[0014] Beneficial effects of the present invention:

[0015] The present invention can reduce the burden on the server side and improve the scalability and fault tolerance of the system by implementing load balancing on the client side. According to the actual load conditions and processing capabilities of the server nodes in the microservice cluster, the load balancing algorithm is dynamically selected, thereby avoiding the problem of resource waste and low efficiency caused by using the dynamic feedback weight load balancing algorithm under low load conditions. In addition, in the dynamic feedback weight load balancing algorithm, the weight coefficient can be dynamically adjusted according to the actual load conditions of the server and the differences in resource occupation by different service types to update the weight; finally, the available server nodes are evaluated according to the weights, and the optimal server node is selected to forward the request. At the same time, the embodiment of the present invention can also take into account the situation where the total amount of system concurrency is large. According to the load conditions of the server nodes, the incoming traffic size is dynamically limited according to a certain strategy at the service entrance, thereby realizing dynamic and real-time traffic control, improving the reliability, scalability, response speed and throughput of the system, and improving the stability of the system. Therefore, the present invention is a beneficial load balancing solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of server-side load balancing;

[0017] Figure 2 is a schematic diagram of client load balancing;

[0018] FIG3 is a flow chart of an implementation of the load balancing method according to an embodiment of the present invention;

[0019] FIG4 is a flow chart of a dynamic weight feedback load balancing algorithm according to an embodiment of the present invention;

[0020] FIG5 is a flow chart of a preferred implementation of the load balancing method according to an embodiment of the present invention;

[0021] FIG6 is a schematic diagram of the service request volume of a server node at any time according to an embodiment of the present invention;

[0022] 7 is a diagram of the load balancing system architecture according to an embodiment of the present invention;

[0023] FIG8 is a flow chart of another load balancing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0025] FIG3 is a flow chart of an implementation of a load balancing method according to an embodiment of the present invention. As shown in FIG3 , the method includes the following steps:

[0026] 101. The client node periodically sends a service request to each server node in the microservice cluster to obtain performance parameter load information of the server node and caches the load information in Redis; the load information includes CPU utilization, memory utilization, network bandwidth utilization, and disk I / O utilization;

[0027] In an embodiment of the present invention, the load balancer in the client node periodically sends a request to each server node to obtain the node's load information. This can be based on the Linux system providing a / proc file system to save the current operating status of the system. The node's load indicator information such as memory utilization and CPU utilization can be obtained by reading and analyzing the / proc file and stored in the Redis cache. Among them, the Redis (Remote Dictionary Server) is an open source, in-memory data structure storage system that can be used as a database, cache, and message middleware. The Redis supports multiple types of data structures, such as strings, hashes, lists, sets, sorted sets, etc., and provides rich operation commands for these data structures.

[0028] For the load balancer, the load balancer establishes a socket connection to the available server nodes and introduces the query rate per second (QPS) parameter. The QPS parameter is the number of queries that a server can respond to per second. The QPS parameter value can be calculated in real time based on the reception and processing volume of the current server node tasks. The server load information collection module collects the CPU utilization, memory utilization, disk I / O utilization and network bandwidth utilization of the server nodes corresponding to the available service instances every T seconds (T is initially set to 5 seconds. When the node QPS exceeds the threshold, the period is set to 2 seconds). The information is pushed to the load balancer through the socket connection and cached in Redis.

[0029] In the initial state, the initial weight is set and Redis caching is performed based on the hardware resource information of each server node in the virtual machine, combined with the initial CPU performance, memory performance, disk I / O and network bandwidth performance of the server in the virtual machine. The initial weight is set based on the hardware resource information of each server node in the virtual machine in the initial state, specifically including:

[0030] Assume that They represent the initial CPU performance, memory performance, disk I / O, and network bandwidth performance of the i-th server node in the microservice cluster, respectively, where i∈(1,2,...,n) and n is the number of server nodes in the microservice cluster. The specific performance indicators are defined as follows:

[0031] Among them, C max 、R max , I max 、N max Respectively represent the maximum number of available CPUs, maximum available memory, maximum available disk, and maximum available network bandwidth. cpu 、U mem 、U io 、U net Respectively represent CPU utilization, memory utilization, disk I / O utilization, and network bandwidth utilization. Represents the sum of the CPU, memory, disk I / O and network bandwidth performance of all server nodes in the cluster. Defined as the initial weight of server node i∈(1,2,...,n).

[0032] Among them, ω c 、ω r 、ω io 、ω nThey represent the weight coefficients corresponding to the performance of CPU, memory, disk I / O, and network bandwidth respectively. k is a constant, which serves as an adjustment factor for the initial weight of the server node.

[0033] 102. The client node makes a judgment based on the actual load rate and the actual load rate threshold of the load information of each server node in the microservice cluster, or / and the size of the comprehensive load rate and the comprehensive load rate threshold. If the load rate exceeds the corresponding threshold, the dynamic feedback weight load balancing algorithm is used to determine the server node with the largest current weight.

[0034] In an embodiment of the present invention, among the available server nodes, when the load rate or the comprehensive load rate of each load information of a server node is greater than the single threshold value and the comprehensive load rate threshold value of each performance parameter set for the server node, a dynamic feedback weighted load balancing algorithm is used to achieve reasonable distribution of requests. Otherwise, some common load balancing algorithms can be used to evenly distribute service requests to appropriate server nodes, including but not limited to polling algorithms, weighted polling algorithms and minimum connection algorithms, etc. For example, a smooth weighted polling load balancing algorithm can be used.

[0035] When the present invention uses a dynamic feedback weight load balancing algorithm to achieve reasonable allocation of requests, as shown in Figure 4, it is first determined whether it is the first time to obtain load information. If so, the weight is calculated based on the load information and saved to Redis. If not, the standard deviation of the node resource utilization rate and the set threshold are introduced. The standard deviation of the node resource utilization rate is compared with the set threshold to determine whether the weight needs to be updated. If the standard deviation of the node resource utilization rate is greater than the set threshold, it means that the current server node is overloaded and the weight needs to be calculated and updated and saved to Redis. Otherwise, the weight does not need to be calculated and updated, and the previous weight information is directly obtained from Redis. By comparing the weights of the available server nodes, the server node with the largest weight is determined. When recalculating the weight, the different resources occupied by different service types (IO intensive and CPU intensive) and the errors in node performance evaluation should be considered, resulting in uneven resource utilization of nodes. By comparing the load evaluation indicators and the mean value of load evaluation index Determine whether the remaining performance load evaluation index weight coefficient needs to be increased or decreased.

[0036] Calculating the weight according to the load information specifically includes:

[0037] Obtain the load information of each server node from Redis, namely the CPU utilization, memory utilization, network bandwidth utilization, and disk I / O utilization of the server node, and convert the resource utilization into resource remaining performance;

[0038] The evaluation function is constructed by combining the four residual performance load evaluation indicators of each server node with the weight coefficient using the weighted sum method, and the dynamic weight of the server node is calculated using the evaluation function;

[0039] The final weight of the server node is obtained by combining the initial weight and dynamic weight of the server node.

[0040] Assume that Wd i Defined as the dynamic weight of server node i, where i∈(1,2,...,n), with P C (Total), P R (Total), P I (Total), P N (Total) represents the total remaining performance of CPU, memory, disk I / O and network bandwidth of all server nodes in the server cluster, respectively. They represent the remaining CPU performance, memory performance, disk I / O performance, and network bandwidth performance of the i-th server in the cluster respectively.

[0041] Among them, ω c 、ω r 、ω io 、ω n Respectively represent the weight coefficients corresponding to the remaining CPU performance, remaining memory performance, remaining disk I / O performance, and remaining network bandwidth performance. V represents the reference variable for weight changes. The default value is 1.

[0042] Let the final weight of the server node be WE i The greater the final weight, the greater the possibility of being assigned to the service request. i Initial weight with server node There is a positive correlation, but It is static. During the service request processing, the remaining performance of the server changes continuously. It can only reflect the performance of a single server node but cannot reflect its load situation. Therefore, the present invention introduces a dynamic weight Wd i , reflects the load of the server node to a certain extent, WE i With Wd i There is also a positive correlation. B is the amplification constant, which amplifies the result several times, reduces the discarded decimal part, and thus reduces the error. i The calculation formula is as follows:

[0043] This embodiment can accurately reflect the performance and load conditions of the server nodes by combining static weights and dynamic weights, making the final weights more accurate and improving the efficiency of load balancing.

[0044] The comparison of the standard deviation of the node resource usage rate with the set threshold to determine whether the weight needs to be updated specifically includes:

[0045] If the collected loads of each server node are weighted and re-assigned frequently each time, not only will the purpose of load balancing fail to be achieved, but the performance of the server nodes will be affected. The standard deviation is used to judge the resource utilization of each server node, and a set threshold is introduced as the critical value for whether to start the weight modification process. By comparing the standard deviation of the node resource utilization rate with the set threshold, it is determined whether the weight needs to be updated. If the standard deviation is greater than the set threshold, the weight is modified, otherwise the weight is not modified. Let the standard deviation of the resource utilization of the server node be S(σ), and the resource utilization of the server node be U i Indicates that the average dynamic utilization of the server node is expressed as If , the standard deviation of the resource utilization of the server node is calculated as:

[0046] in,

[0047] In the embodiment of the present invention, considering the problem of uneven resource utilization of nodes caused by different service types (IO-intensive and CPU-intensive) when requesting and errors in node performance evaluation, determining whether the load evaluation index weight coefficient needs to be increased or decreased specifically includes:

[0048] During the actual operation of a cluster system, resource utilization on server nodes can be uneven due to varying resource usage by different requests and errors in node performance evaluation. Consider two types of services: I / O-intensive and CPU-intensive. When requests are I / O-intensive, the physical machine needs to perform a large number of fast disk I / O reads, placing high demands on disk I / O. Therefore, the weights for disk I / O utilization and network bandwidth utilization should be increased. When requests are CPU-intensive, the physical machine needs to provide rapid computing power, so the weights for CPU utilization and memory utilization should be increased.

[0049] Considering that a higher load evaluation index should obtain a higher weight coefficient, by comparing the load evaluation index and the mean value of load evaluation index Determine whether the load evaluation index weight coefficient needs to be increased or decreased, where The following weight coefficient ωc The update of is illustrated as an example.

[0050] when It is considered that the CPU load evaluation index of server node i has an increasing impact on the server, and its corresponding weight coefficient ω should be increased. c Otherwise, it is considered that the CPU load evaluation index of server node i has less influence on the server, and its corresponding weight coefficient ω should be reduced. c , the weight coefficient correction variable α is shown as follows:

[0051] The updated weight coefficient expression is as follows:

[0052] Since the weight coefficients need to satisfy the requirement that the sum of the weight coefficients is 1, this embodiment can be used to calculate ω c '、ω r '、ω io '、ω n 'Perform the following normalization:

[0053] In some embodiments of the present invention, when the difference between the actual load rate of each server node and the actual load rate threshold, or / and the difference between the comprehensive load rate and the comprehensive load rate threshold are judged, if the corresponding load rate threshold is not exceeded, a traditional load balancing algorithm can be used for load balancing. Taking the smooth weighted round-robin algorithm as an example, first, the sum of the weights of each server is calculated. Initially, the current effective weights of all server nodes are set to 0. As requests arrive, each time the server node with the largest current effective weight is selected from all server nodes for processing, after processing the request, the effective weight of the selected server node needs to be subtracted from the sum of the weights of all selected server nodes to update the current effective weight of the server node. At the next request, the current effective weights of all server nodes are recalculated, and the server node with the largest current effective weight is selected. This cycle is repeated, and smoothing is performed during the weighting process to avoid server nodes with large weights being continuously selected, causing excessive load in a local short period of time, thereby achieving load balancing and improving the performance and stability of the entire system.

[0054] 103. The client node sends a call request to the server node with the largest current weight in the microservice cluster, executes the request in the corresponding microservice execution module, and returns the execution result to the client node.

[0055] In an embodiment of the present invention, the client node sends a call request to the server node with the maximum weight determined by the dynamic feedback weight load balancing algorithm, executes the request in the microservice execution module of the server node, and returns the execution result to the client node; thereby achieving the purpose of load balancing.

[0056] FIG5 is a flow chart of a preferred implementation of the load balancing method according to an embodiment of the present invention. As shown in FIG5 , the method includes the following steps:

[0057] 201. The client node periodically sends a service request to each server node in the microservice cluster to obtain performance parameter load information of the server node and caches the load information in Redis; the load information includes CPU utilization, memory utilization, network bandwidth utilization, and disk I / O utilization;

[0058] 202. The client node makes a judgment based on the difference between the actual load rate and the actual load rate threshold, or / and the difference between the comprehensive load rate and the comprehensive load rate threshold of the load information of each server node in the microservice cluster. If the load rate threshold is exceeded, the client node uses a dynamic feedback weight load balancing algorithm to determine the server node with the largest current weight.

[0059] 203. The client node sends a call request to the server node with the largest current weight in the microservice cluster, executes the request in the corresponding microservice execution module, and returns the execution result to the client node.

[0060] 204. If the CPU utilization and / or memory utilization of the server node periodically obtained by the client node exceeds the corresponding preset threshold, a flow control strategy is adopted to adjust the deviation between the time when each client node first sends a service request and the system start time, on the premise of minimizing the amount of service requests received by the server node within a certain period of time.

[0061] This embodiment will focus on step 204. In this embodiment, during initial system operation, the flow control level is set to level 1. At this level, normal requests do not require flow control. The flow control level is dynamically adjusted based on the CPU or memory usage of the server node, serving as a reference for flow control adjustment, to achieve varying degrees of control effectiveness.

[0062] In this embodiment, the incoming traffic size is dynamically limited according to a certain strategy to reduce the load pressure of the node, mainly including when the CPU occupancy rate U cpu Or memory usage U mem When the rate exceeds 90%, the flow control level is set to level 2. Assume that there are m consumer sets {c1, c2, ..., c mEach consumer periodically sends a service request to the server node, and the period is T 1 ,T 2 ,...,T m The consumer is also the client node c x The time to send the service request is respectively Indicates that, let c x The deviation between the time when the service request is first sent and the time when the system starts is Δt x , then c x The deviation time of sending the jth service request relative to the system start time is:

[0063] Since the accuracy of the clock hardware of each device in the business system is different and there is delay jitter in network transmission, there may be a deviation between the time when each consumer sends a business request and the time when the request arrives at the server node. To simplify the model, let c x The time when the jth service request arrives at the server node meets the interval Among them, δ x For client node c x The corresponding service delay parameter is the same as the consumer c x It is related to factors such as the network status between server nodes, which is a constant and satisfies 0<δ x <T y .

[0064] Assumption c x The jth service request arrives at If the number of service requests is evenly distributed within the interval, the service request volume of the server node at any time is shown in Figure 6 (assuming 3 consumers, that is, 3 client nodes). It can be expressed as the sum of the projections of the arrival time intervals of each consumer's request on the server time axis. That is, the number of service requests on the server at any time can be expressed as:

[0065] The client node c x The indicator function Φ(t,x) at time t is defined as:

[0066] when When Φ(t,x)=1, otherwise Φ(t,x)=0, in order to balance the service load of the server node, that is, the number of service requests received by the server node within τ time is as small as possible, under the premise that the consumer service request cycle remains unchanged, the Δt of each consumer can be used to balance the service load of the server node. x Adjustments are made to avoid multiple consumer service requests from arriving in the same time period. The design optimization objective function is:

[0067] st0≤Δt x <T y

[0068] Where (Δt 1 ,Δt 2 ,...,Δt m ) is the time deviation parameter vector of all consumers in the business system, i.e., m client nodes.

[0069] In some embodiments of the present invention, a genetic algorithm may be used to solve the above optimization problem. The genetic algorithm can find the optimal solution through excellent global search capabilities. Specifically, the solution can be achieved by following the following steps:

[0070] (1) Initialize the population: define the population size as K, initialize K groups of solutions of the objective function, each group of solutions is called an individual or chromosome, and the components in each group of solutions are called genes, that is, time deviation parameter vectors, where each parameter satisfies 0≤Δt x <T y Constraints;

[0071] (2) Calculation of fitness: When calculating the fitness value of each individual, we should follow the principle that the larger the fitness value, the better the individual. The fitness value can be obtained by the objective function Take the reciprocal to express;

[0072] (3) Selection operation: Use the roulette wheel method to select a portion of excellent individuals as parents for reproducing the next generation based on their fitness values;

[0073] Based on the roulette wheel method, that is, the selection strategy based on the fitness ratio. Define F i is the fitness value of individual i,

[0074] The probability p that individual i is selected i for:

[0075] Compare the random number rand in the interval [0,1] with the cumulative probability q of individual i in turn i ,in Select q i The first individual whose value is greater than rand is repeated K times until K individuals are selected.

[0076] (4) Crossover operation: Two individuals are randomly selected from the population based on the crossover probability and the gene positions of the crossover are exchanged to generate new individuals;

[0077] Using real number crossover, let b be a random number in the interval [0,1], and the kth chromosome a k and chromosome a lThe crossover operation method at position j is as follows:

[0078] (5) Mutation operation: mutation operation is performed on the newly generated offspring individuals. An individual and the mutation gene position are randomly selected in the population based on the mutation probability and non-uniform mutation is performed. The mutation operation can increase the diversity of the population by changing the values ​​of certain parameters in the individual;

[0079] Let r be a random number in the interval [0,1], gen be the current number of iterations, maxgen be the maximum number of iterations, and the jth gene a of the i-th individual be ij The method for performing mutation operations is as follows:

[0080] where a min and a max A ij The lower and upper bounds of .

[0081] (6) Update population: merge parent and offspring individuals into a new generation population;

[0082] (7) Repeat steps (2) to (6): Set the evolutionary generation number to maxgen, and iterate steps (2) to (6) maxgen times until the termination condition is reached;

[0083] (8) Output the optimal solution: When the termination condition is met, the individual with the largest fitness value is output as the optimal solution, that is, the time deviation parameter vector (Δt 1 ,Δt 2 ,...,Δt m ).

[0084] Through the above steps, the present invention can find a way to minimize the number of services arriving within a period of time, thereby ensuring stable operation of the server.

[0085] In some embodiments of the present invention, the above-mentioned optimization problem can also be solved by other existing target optimization algorithms such as ant colony algorithm, beetle beard algorithm, bee colony algorithm and particle swarm algorithm. The client node can solve the optimization problem by calling a processor equipped with an operation program embedded with the above-mentioned optimization algorithm.

[0086] Figure 7 is an architecture diagram of a microservice dynamic adaptive client load balancing system. As shown in Figure 7, the system includes at least a client, a server and a registration center; the client includes a client node, a service gateway and a load balancer composed of a device / gateway and a management platform; the server includes a microservice cluster, and the microservice cluster includes multiple server nodes, and multiple server nodes constitute a microservice cluster; the server node includes a server load information collection module and a microservice execution module; the client node and the server node complete registration in the registration center respectively; the client node initiates a request to each server node in the microservice cluster through the device / gateway and the service gateway; the server node collects the load information of the server node through the server load information collection module, processes the collected load information through the dynamic feedback weight load balancing algorithm equipped with the load balancer, and determines the server node with the current largest weight; sends a call request to the server node with the current largest weight in the microservice cluster, executes the request in the corresponding microservice execution module, and returns the execution result to the client node.

[0087] In an embodiment of the present invention, the client is composed of a client node, which is a user of the microservice, including a device / gateway and a management platform. The client node is responsible for initiating a request or making multiple requests in a concurrent manner, for example, initiating a message queue telemetry transport protocol (MQTT) request to the service gateway through the device / gateway, and initiating a hypertext transfer protocol (HTTP) request to the service gateway through the management platform. The service registration and discovery component Nacos can be used to obtain information such as the IP address, port number and health status of the microservice instance, and the load balancing algorithm can be dynamically selected by the load balancer to select the most appropriate microservice instance to process the request to achieve the purpose of load balancing. The service gateway can filter and securely verify the routing request and forward the request initiated by the client node to the load balancer. The service gateway can forward the request to different microservice instances based on factors such as the type, traffic, performance and resources of the service request, thereby reducing the pressure on a single microservice instance, improving the availability and performance of the system, ensuring load balancing, and improving the stability and performance of the system.

[0088] In an embodiment of the present invention, the load balancer may further include a load information processing module, which is a core component of the microservice architecture. It first receives the request filtered and routed by the service gateway, and quickly retrieves which cluster servers have deployed the requested service based on the service address provided by the registration center. Then, based on the resource monitoring data, it calculates the load of each cluster server and selects the server with the largest weight. Finally, it evenly distributes the request to multiple instances to achieve a load balancing effect and improve the performance and reliability of the entire system.

[0089] The registration center is mainly responsible for the registration and service discovery of microservices. The service provider, that is, the server node, registers its own information with the registration center and forms corresponding registration information, which at least includes the service name of the server node. The service consumer, that is, the client node, discovers available microservice instances through the service name, queries available microservice instances and selects the appropriate instance for call according to the load balancing algorithm. The registration center provides a service address for quickly retrieving instance information, provides the client's load balancer with necessary data, helps realize the basic functions of the load balancer, and improves the reliability and performance of the entire microservice architecture. Among them, the registration center can be a Nacos registration center, which can quickly realize functions such as service discovery, service configuration, service metadata and traffic management.

[0090] In an embodiment of the present invention, the server is composed of a microservice cluster, which is composed of multiple server nodes deployed with microservice instances, and mainly includes two parts: a microservice execution module and a server load information collection module;

[0091] Furthermore, the main function of the microservice execution module is to receive and process the load request sent by the load balancing scheduler from the client's load balancer, and select a server node from the cluster to execute the request according to the preset load balancing algorithm. In addition, the microservice execution module also needs to receive the response returned by the service request and feed it back to the requesting client. Since microservice instances are usually deployed on different servers, the microservice execution module needs to forward requests between different server nodes to ensure that the task can be executed and processed in a timely manner.

[0092] Furthermore, the server load information collection module periodically collects heterogeneous server cluster load parameter indicators. Considering that the selected evaluation server resource load parameter indicators need to be easy to collect, they will not cause excessive resource consumption to the service, and can also accurately measure the performance status of the server and the server node's processing ability for service requests. Based on the above considerations, CPU utilization, memory utilization, disk I / O utilization and network bandwidth utilization hardware indicators are selected to define the comprehensive load, thereby comprehensively considering the load processing performance of each microservice cluster to limit request allocation.

[0093] FIG8 is a flow chart of another microservice dynamic adaptive client load balancing method provided by an embodiment of the present invention. As shown in FIG8 , the method includes the following steps:

[0094] Step 1: Initialize the performance parameter settings and initial weight settings of each server node, and set the flow control level to level 1;

[0095] Step 2: The consumer, or client node, initiates a service request and passes the request to a service gateway, such as Spring Cloud Gateway, and then goes to step 3.

[0096] Step 3: The service gateway accepts the request and sends it to the load balancing module, then goes to step 4;

[0097] Step 4: The registration center has recorded the relevant information of the microservice. After it is transmitted to the load balancing module, the load balancing module queries the server node where the service instance is deployed in the registration center. If the service is not deployed, it returns that the request cannot be executed and ends the process. If the service is deployed, it finds the list of available server nodes for the service instance corresponding to the service request at this time, obtains its IP address, port and other information, and sends it to the load balancing module. Then it continues to execute and goes to step 5.

[0098] Step 5: Determine whether there is only one server node in the microservice cluster that has the service deployed. If there is only one server node that has the service deployed, the service deployed on that server node will directly execute the service request. If there are more than one server node that has the service deployed, a comparison is required to determine which server node will ultimately execute the request, and then proceed to step 6.

[0099] Step 6: The load information processing module periodically obtains and calculates the current load information of the required server node. When the CPU usage or memory usage exceeds 90%, the flow control level is set to level 2, and the incoming traffic size is dynamically limited according to a certain strategy, and then the process goes to step 7.

[0100] Step 7: Among the available server nodes, when the load ratio or the combined load ratio of each load information of a server node is greater than the individual load information threshold and the combined load ratio threshold set for the server node, the dynamic feedback weighted load balancing algorithm is used, and the process goes to step 8. The core of the dynamic feedback weighted load balancing algorithm is to use weight as the criterion for selecting a server. Otherwise, the smooth weighted round-robin algorithm is used, and the process goes to step 9.

[0101] Step 8: Using the dynamic feedback weight load balancing algorithm, the load balancing module sends a call request to the server node with the largest weight, and then goes to step 10;

[0102] Step 9: Use a smooth weighted round-robin algorithm to evenly distribute service requests to appropriate server nodes, and then go to step 10;

[0103] Step 10: The microservice execution module in the server cluster distributes the requests reasonably, receives the responses returned by the service requests and feeds them back to the requesting client node, thereby achieving the purpose of load balancing; the process can be ended directly or wait for the next service request from the client node and start from step 2.

[0104] It can be understood that a microservice dynamic adaptive client load balancing method and system in the embodiments of the present invention belong to the same inventive concept, and their corresponding technical features can be referenced to each other, so the present invention will not repeat them.

[0105] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A microservice dynamic adaptive client load balancing method, characterized in that: The method comprises: The client node periodically sends service requests to each server node in the microservice cluster to obtain the load information of the server node and cache the load information in Redis; the load information includes CPU utilization, memory utilization, network bandwidth utilization, and disk I / O utilization; The client node makes a judgment based on the actual load rate and the actual load rate threshold of the load information of each server node in the microservice cluster, or / and the comprehensive load rate and the comprehensive load rate threshold. If the load rate threshold is exceeded, the dynamic feedback weight load balancing algorithm is used to determine the server node with the largest current weight. The client node sends a call request to the server node with the largest current weight in the microservice cluster, executes the request in the corresponding microservice execution module, and returns the execution result to the client node.

2. According to claim 1, a microservice dynamic adaptive client load balancing method is characterized in that: The method of using a dynamic feedback weight load balancing algorithm to determine the server node with the largest current weight includes: Get the load information and determine whether it is the first time to obtain it. If so, calculate the weight based on the load information and save it to Redis; if not, compare the standard deviation of the resource utilization rate of the current server node and the set threshold. If the standard deviation of the resource utilization rate of the current server node is greater than the set threshold, calculate the weight based on the load information and save it to Redis. Otherwise, directly get the weight of the available server nodes from Redis, and determine the server node with the largest weight by comparing the weights of the available server nodes.

3. According to claim 2, a microservice dynamic adaptive client load balancing method is characterized in that: The calculating of the weight according to the load information specifically includes: Obtain the load information of each server node from Redis and convert the resource utilization into the remaining resource performance; The evaluation function is constructed by combining the four remaining performance load evaluation indicators of each server node with the weight coefficient using the weighted sum method, and the dynamic weight of the server node is calculated using the evaluation function; The initial weight and dynamic weight of the server node are combined to obtain the final weight of the server node.

4. A microservice dynamic adaptive client load balancing method according to claim 3, characterized in that: The expression of the dynamic weight of the server node is as follows: Among them, Wd i represents the dynamic weight of server node i; V represents the reference variable of weight change; ω c ,ω r ,ω io ,ω n Respectively represent the weight coefficients corresponding to the remaining CPU performance, remaining memory performance, remaining disk I / O performance, and remaining network bandwidth performance; Respectively represent the CPU surplus performance and memory surplus of server node i in the microservice cluster performance, disk I / O remaining performance, and network bandwidth remaining performance; C (Total), P R (Total), P I (Total), P N (Total) represents the total remaining performance of CPU, memory, disk I / O, and network bandwidth of all server nodes in the microserver cluster.

5. A microservice dynamic adaptive client load balancing method according to claim 4, characterized in that: The expression of the final weight of the server node is as follows: Among them, WE i represents the final weight of server node i; B is the amplification constant; Represents the initial weight of server node i.

6. A microservice dynamic adaptive client load balancing method according to claim 3, characterized in that: The weight coefficients corresponding to the remaining performance of the server nodes are modified according to the performance evaluation index. The modified weight coefficients are as follows: Among them, ω c ′、ω r ′、ω io ′、ω n ′ represents the weight coefficients of the corrected CPU remaining performance, memory remaining performance, disk I / O remaining performance, and network bandwidth remaining performance respectively; They respectively represent the CPU remaining performance load evaluation index, memory remaining performance load evaluation index, disk I / O remaining performance load evaluation index, and network bandwidth remaining performance load evaluation index of server node i in the microservice cluster; They respectively represent the mean value of the CPU remaining performance load evaluation index, the mean value of the memory remaining performance load evaluation index, the mean value of the disk I / O remaining performance load evaluation index, and the mean value of the network bandwidth remaining performance load evaluation index of the server node i in the microservice cluster.

7. A microservice dynamic adaptive client load balancing method according to claim 1, characterized in that: The method also includes, if the CPU utilization and / or memory utilization of the server node periodically obtained by the client node exceeds the corresponding preset threshold, then, on the premise of minimizing the amount of service requests received by the server node within a certain period of time, using a flow control strategy to adjust the deviation between the time when each client node first sends a service request and the system start time.

8. A microservice dynamic adaptive client load balancing method according to claim 7, characterized in that: The optimization model for minimizing the number of business requests received by the server node within a certain period of time is expressed as: st0≤Δt x <T y Among them, τ represents a certain time, Φ(t,x) represents the client node c x The indicator function at time t is When Φ(t,x)=1, otherwise Φ(t,x)=0, (Δt 1 ,Δt 2 ,...,Δt m ) is the business system There are consumers, i.e., m client nodes, with a time deviation parameter vector, T y represents the yth period, δ x For client node c x The corresponding service delay parameters, Represents the client node c x The deviation time of sending the jth service request relative to the system start time.

9. A microservice dynamic adaptive client load balancing system, used to implement a microservice dynamic adaptive client load balancing method as claimed in claim 1, characterized in that: The system at least includes a client, a server and a registration center; the client includes a client node, a service gateway and a load balancer composed of a device / gateway and a management platform; the server includes a microservice cluster, and the microservice cluster includes multiple server nodes; the server node includes a server load information collection module and a microservice execution module; The client node and the server node complete registration in the registration center respectively; The client node initiates a request to each server node in the microservice cluster through the device / gateway and the service gateway; The server node collects the load information of the server node through the server load information collection module, processes the collected load information through the dynamic feedback weight load balancing algorithm carried by the load balancer, and determines the server node with the largest current weight; Send a call request to the server node with the largest current weight in the microservice cluster, execute the request in the corresponding microservice execution module, and return the execution result to the client node.

Images