A gateway service dynamic load balancing method, device, equipment and storage medium

Abstract

The application relates to the technical field of gateway service, and discloses a gateway service dynamic load balancing method, device, equipment and storage medium, the method comprises the following steps: matching a corresponding upstream service subset according to a received user request; determining a load balancing strategy based on the matched upstream service subset, wherein a matching relationship between the load balancing strategy and the upstream service subset is pre-stored into a local cache through a self-defined resource declaration, an algorithm rule corresponding to the load balancing strategy is loaded into a local rule cache, and a change event of the load balancing strategy is monitored in real time; when the change event of the load balancing strategy is monitored, the algorithm rule corresponding to the load balancing strategy is updated in the local rule cache according to the change event; a target address is obtained by executing a rule algorithm corresponding to the load balancing strategy, and the user request is sent to a target service instance based on the target address. The application can improve the load balancing flexibility in a super-large-scale cluster scenario.

Description

Technical Field

[0001] This invention relates to the field of gateway service technology, and specifically to a dynamic load balancing method, apparatus, device, and storage medium for gateway services. Background Technology

[0002] With the widespread adoption of cloud computing, the deployment and operation of containerized applications have become increasingly important. Against this backdrop, Kubernetes (hereinafter referred to as K8s), as an open-source container orchestration platform, has become the de facto standard for deploying and operating containerized applications.

[0003] In a Kubernetes cluster, the gateway service (kube-gateway) is a gateway service that provides Layer 7 load balancing and traffic control, which can significantly reduce the load pressure on API services, such as... Figure 1 As shown, currently in Kubernetes clusters, the gateway service is not designed to support plug-in load balancing capabilities. The load balancing logic is integrated through hard coding, which is inefficient and does not support dynamic expansion, resulting in insufficient flexibility in ultra-large-scale cluster scenarios. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device and storage medium for dynamic load balancing of gateway services to solve the technical problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios.

[0005] In a first aspect, the present invention provides a dynamic load balancing method for gateway services, comprising:

[0006] Match the corresponding upstream service subset based on the received user request;

[0007] The load balancing strategy is determined based on the matched upstream service subset. The matching relationship between the load balancing strategy and the upstream service subset is pre-stored in the local cache through a custom resource declaration. The algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the load balancing strategy change events are monitored in real time. When a load balancing strategy change event is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event.

[0008] By executing the rule algorithm corresponding to the load balancing strategy, the target address is obtained, and the user request is sent to the target service instance based on the target address.

[0009] The gateway service dynamic load balancing method of this invention loads load balancing strategies through custom resource declarations, loads the corresponding algorithm rules of the load balancing strategies into a local rule cache, and monitors load balancing strategy change events in real time. When a load balancing strategy change event is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. It can dynamically update the algorithm rules corresponding to the load balancing strategies in each upstream service subset in the local rule cache according to the real-time monitored change events, thereby adjusting the load distribution. This can solve the problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios and improve load balancing efficiency.

[0010] Furthermore, the load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy;

[0011] Correspondingly, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, including:

[0012] When the load balancing strategy is the first type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the local algorithm library through a lightweight interpreter;

[0013] When the load balancing strategy is the second type of load balancing strategy, a lightweight interpreter is used to match the algorithm rules corresponding to the load balancing strategy from the external source code and perform rule transformation.

[0014] In this approach, the algorithm rules corresponding to the load balancing strategy are loaded based on a lightweight interpreter, enabling the load strategy to support hot expansion. It can not only match algorithm rules from the local algorithm library, but also match corresponding rules from external source code and perform rule conversion, making it convenient to expand the algorithm content.

[0015] Furthermore, the execution results obtained from executing the first preset algorithm rule are pre-cached in the local cache. When the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

[0016] In this approach, by pre-caching the execution results of the first preset algorithm rule, calculations can be avoided after receiving user requests in certain situations, thereby improving the system's processing speed and efficiency and reducing customer waiting latency.

[0017] Furthermore, based on the change event, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache, including:

[0018] Determine the type of change event;

[0019] If the change event is a creation event or an update event, then check whether the currently created or updated policy has been added to the local rule cache. Then, perform rule overwriting or rule merging in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing policy. If the change event is a deletion event, then remove the rule algorithm corresponding to the deleted load balancing policy from the local rule cache.

[0020] In this approach, the local rule cache is dynamically updated based on load balancing policy change events, ensuring timely updates of the load balancing policy. The local rule cache is dynamically updated based on the service discovery mechanism of monitoring, eliminating the need for manual intervention and reducing operation and maintenance costs while improving system stability.

[0021] Furthermore, after sending the user request to the target service instance based on the target address, the process also includes:

[0022] Periodically access the service instance status monitoring interface to obtain service instance status information;

[0023] The rule algorithm for adjusting the load balancing strategy based on service instance status information.

[0024] In this approach, the accuracy of the rule algorithm execution results is improved by monitoring the status information of service instances and adjusting the rule algorithm corresponding to the load balancing strategy accordingly.

[0025] Furthermore, after obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, the process also includes:

[0026] If the target address does not exist, the second preset rule algorithm is executed, and the target address is updated according to the execution result of the second preset rule algorithm.

[0027] The user request is sent to the target service instance based on the updated target address.

[0028] In this approach, when the target address does not exist, the target address can be updated by executing a second preset rule algorithm, ensuring that user requests can be correctly sent to the target service instance, thus improving the robustness of the system. Even in the face of changes or unavailability of the target address, the correct routing of requests can be ensured.

[0029] In a second aspect, the present invention provides a gateway service dynamic load balancing device, comprising:

[0030] The instance set matching module is used to match the corresponding upstream service subset based on the received user request.

[0031] The strategy determination module is used to determine the load balancing strategy based on the matched upstream service subset. The matching relationship between the load balancing strategy and the upstream service subset is pre-stored in the local cache through a custom resource declaration. The algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the change events of the load balancing strategy are monitored in real time. When a change event of the load balancing strategy is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event.

[0032] The strategy execution module is used to obtain the target address by executing the rule algorithm corresponding to the load balancing strategy, and then send the user request to the target service instance based on the target address.

[0033] Thirdly, the present invention provides a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the gateway service dynamic load balancing method described in the first aspect or any corresponding embodiment thereof.

[0034] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the gateway service dynamic load balancing method described in the first aspect or any corresponding embodiment thereof.

[0035] Fifthly, the present invention provides a computer program product including computer instructions for causing a computer to perform a document processing method as described in the first aspect or any of its corresponding embodiments. Attached Figure Description

[0036] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating the dynamic load balancing method for gateway services in related technologies;

[0038] Figure 2 This is a flowchart illustrating a dynamic load balancing method for gateway services according to an embodiment of the present invention.

[0039] Figure 3 This is a flowchart illustrating another gateway service dynamic load balancing method according to an embodiment of the present invention;

[0040] Figure 4This is a schematic diagram of the initialization process of the gateway service dynamic load balancing method according to an embodiment of the present invention;

[0041] Figure 5 This is a flowchart illustrating the cyclic tuning process of the gateway service dynamic load balancing method according to an embodiment of the present invention.

[0042] Figure 6 This is a flowchart illustrating the dynamic load balancing method for gateway services in an embodiment of the present invention for processing user requests.

[0043] Figure 7 This is a structural block diagram of the gateway service dynamic load balancing device according to an embodiment of the present invention;

[0044] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The dynamic load balancing method for gateway services in this invention is applicable to ultra-large-scale clusters, typically with more than 5,000 nodes. Through the extended architecture, it can support K8S clusters with millions of containers.

[0047] After the gateway service of the Kubernetes cluster receives a user request, it will analyze the situation of the upstream service instance and forward the request to the corresponding API service (kube-apiserver). The API service is the traffic entry point of the Kubernetes cluster, providing access control, authentication, and permission management functions, and accepting client requests for resource query, creation, update, and deletion. The API service connects the requests to the corresponding service instance to achieve load balancing.

[0048] Based on related technologies, gateway services are not designed to support pluggable load balancing capabilities. Load balancing logic is integrated through hard-coded (round-robin) methods, resulting in low efficiency and lack of dynamic expansion support. This leads to insufficient flexibility in complex business scenarios and high operational costs. This invention provides a dynamic load balancing method for gateway services.

[0049] According to an embodiment of the present invention, a dynamic load balancing method for gateway services is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0050] This embodiment provides a dynamic load balancing method for gateway services, which can be used in gateway devices, specifically applied to the load balancer controller within the gateway device. Figure 3 As shown, the load balancer controller is the core of the entire load balancing solution and has the following functions: listening for changes in load balancing policies and updating the local rule cache synchronously; loading various custom algorithm rules through the plugin manager and converting request addresses into service instance addresses that can be proxied; periodically accessing the service instance status monitoring interface to synchronize the availability status and capacity information of service instances in real time, which is used as the basis for judging service degradation, rate limiting and circuit breaking.

[0051] like Figure 2 As shown, the dynamic load balancing method for gateway services in this embodiment of the invention includes the following steps:

[0052] Step S101: Match the corresponding upstream service subset based on the received user request.

[0053] Specifically, users send user requests through the client to obtain the corresponding services.

[0054] The user request includes information such as the user's domain name and port. When the user request is detected, the upstream service subset to which the domain name belongs is matched based on its domain name. The upstream service subset includes multiple service instances. After the load balancer performs load balancing, the user request is sent to one of the service instances for processing.

[0055] Step S102: Determine the load balancing strategy based on the matched upstream service subset. The matching relationship between the load balancing strategy and the upstream service subset is pre-stored in the local cache through a custom resource declaration. The algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache. The change events of the load balancing strategy are monitored in real time. When a change event of the load balancing strategy is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event.

[0056] Specifically, the local cache includes a set of algorithm rule chains sorted by priority. The algorithm rules can be based on the minimum number of connections, the minimum resource consumption, etc., and each algorithm rule is associated with a processing function.

[0057] Load balancing strategies are configured for each upstream service subset using Custom Resource Definitions (CRDs). These CRDs specify the applicable objects of the load balancing strategy (e.g., different sub-clusters, different service instances within the same sub-cluster, or the same service instance within the same sub-cluster), the effective time, and the corresponding algorithm rules. After configuring the load balancing strategy, each upstream service subset caches the algorithm rules from its respective load balancing strategy. Multiple algorithm rules are logically combined to form corresponding rule chains. This custom resource declaration enables dynamic load balancing strategies and supports customized extensions for various business scenarios.

[0058] Furthermore, based on event monitoring of load balancing policy change events, when a change event is detected, such as the deletion, addition, or modification of the load balancing policy, the algorithm rules corresponding to the load balancing policy are deleted, added, or replaced in the local rule cache according to the content of the change event.

[0059] In related technologies, obtaining load balancing strategy rules adopts periodic polling (regardless of whether the configuration changes), which is very performance-intensive in ultra-large-scale scenarios. However, this invention is based on event-driven, and only performs hot reloading on the configuration that has changed, reducing the overhead caused by system polling.

[0060] Step S103: Obtain the target address by executing the rule algorithm corresponding to the load balancing strategy, and send the user request to the target service instance based on the target address.

[0061] Specifically, such as Figure 6 As shown, since the load balancer has converted the load balancing strategy into a rule algorithm and formed a rule chain based on the rule algorithm and stored it in the cache, the target address of the target service instance of the upstream service can be calculated by executing the corresponding rule chain, and the user request can be sent to the target service instance for processing through the target address.

[0062] If no load balancing strategy is configured in the upstream service subset to which the domain belongs, load balancing will be performed according to a pre-set strategy, such as a round-robin strategy.

[0063] The gateway service dynamic load balancing method of this invention loads load balancing strategies through custom resource declarations, loads the corresponding algorithm rules of the load balancing strategies into a local rule cache, and monitors load balancing strategy change events in real time. When a load balancing strategy change event is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. It can dynamically update the algorithm rules corresponding to the load balancing strategies in each upstream service subset in the local rule cache according to the real-time monitored change events, thereby adjusting the load distribution. This can solve the problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios and improve load balancing efficiency.

[0064] The gateway service dynamic load balancing method of this invention implements dynamic load balancing strategy through custom resource declaration, supports custom extensions for various business scenarios, and is event-driven, only hot-loading the configuration that changes, reducing the overhead of system polling.

[0065] In some embodiments, the load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy;

[0066] Correspondingly, in step S102, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, including:

[0067] Step S1021: When the load balancing strategy is the first type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the local algorithm library through a lightweight interpreter.

[0068] Step S1022: When the load balancing strategy is the second type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the external source code by a lightweight interpreter and the rules are converted.

[0069] Specifically, the load balancer controller plugin manager integrates a lightweight interpreter. The plugin manager loads the corresponding plugins based on configuration or command-line parameters.

[0070] The system loads supported load balancing plugins and their algorithm libraries through the plugin manager, and converts the rules defined by the load balancing plugins into a series of intermediate processing steps using a lightweight interpreter. Each algorithm rule serves as a load balancing plugin.

[0071] The type of load balancing strategy is declared in the custom resource declaration. Correspondingly, the load balancing plugin supports two methods: predefined and runtime loading.

[0072] The predefined method involves mounting the local algorithm library (i.e., code files) of the load balancing algorithm to a fixed directory. When the load balancing strategy is the first type of load balancing strategy, the lightweight interpreter performs algorithm rule matching from the local algorithm library, retrieves the algorithm files, and then dynamically runs the load algorithm through the lightweight interpreter.

[0073] Runtime loading, when the load balancing strategy is the second type of load balancing strategy, uses Configmap as a carrier to import external custom load balancing algorithm source code (i.e., external source code). The external source code is submitted as a load balancing plugin to a lightweight interpreter for real-time rule conversion.

[0074] The system retrieves the supported algorithm rules for the load balancing strategy based on the configured local or external algorithm library and converts them into hook functions and their input parameters. If the algorithm rule does not exist, an error message is returned.

[0075] In this approach, the algorithm rules corresponding to the load balancing strategy are loaded based on a lightweight interpreter, enabling the load strategy to support hot expansion. It can not only match algorithm rules from the local algorithm library, but also match corresponding rules from external source code and perform rule conversion, making it convenient to expand the algorithm content.

[0076] In some embodiments, the execution results obtained by executing the first preset algorithm rule are pre-cached in the local cache. When the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

[0077] Specifically, the local cache of the load balancer controller is a set of load rule chains sorted by priority, with each rule associated with a processing function. To improve load efficiency, a first preset algorithm rule is set for specific algorithm scenarios, such as load rules like minimum number of connections or minimum resource consumption. The next access address is pre-calculated based on the first preset algorithm rule to avoid increasing client request latency due to the calculation process.

[0078] In this approach, by pre-caching the execution results of the first preset algorithm rule, calculations can be avoided after receiving user requests in certain situations, thereby improving the system's processing speed and efficiency and reducing customer waiting latency.

[0079] In some embodiments, step S102, updating the algorithm rules corresponding to the load balancing strategy in the local rule cache according to the change event, includes:

[0080] Step S1023: Determine the type of change event;

[0081] Step S1024: If the change event is a creation event or an update event, check whether the currently created or updated strategy has been added to the local rule cache. Then, perform rule overwriting or rule merging in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing strategy. If the change event is a deletion event, remove the rule algorithm corresponding to the deleted load balancing strategy from the local rule cache.

[0082] Specifically, the load balancer controller also includes a tuner, which updates the algorithm rules of the local cache by listening for load balancing policy change events, such as add time, delete time, and update events.

[0083] like Figure 4As shown, before deploying the gateway service, a load balancing policy object is first registered in the cluster where the control plane resides. When the gateway service starts, it loads the load balancing controller, which includes a tuner and a plugin manager. The tuner updates the locally cached load balancing rules by listening for load balancing policy change events (addition, deletion, update), while the plugin manager loads the corresponding plugins based on configuration or command-line parameters.

[0084] like Figure 5 As shown, when the gateway control plane creates, updates, or deletes a load balancing policy, the load balancer controller automatically enters a cyclic tuning process. When creating or updating a load balancing policy, the load balancer controller first checks whether the algorithm rules defined in the load balancing policy have been added to the local rule chain. If they exist and the load balancing policy's merging strategy is overriding, the algorithm rules from the object's custom resource declaration are extracted and directly overwritten in the cache; otherwise, policy merging is performed. When deleting a load balancing policy, the load balancer controller removes the algorithm rules and their processing functions from the cache.

[0085] In addition, deleting a load balancing policy will trigger the load balancer to remove the corresponding algorithm rule. If the corresponding algorithm rule is being referenced, deletion will be prohibited.

[0086] In this approach, the local rule cache is dynamically updated based on changes to the monitored load balancing strategy, ensuring timely updates to the load balancing strategy. The local rule cache is dynamically updated based on the service discovery mechanism of monitoring, eliminating the need for manual maintenance, reducing operation and maintenance costs, and improving system stability.

[0087] Furthermore, in step S103, after sending the user request to the target service instance based on the target address, the following is also included:

[0088] Step S104: Periodically access the service instance status monitoring interface to obtain service instance status information;

[0089] Step S105: Adjust the rule algorithm corresponding to the load balancing strategy based on the service instance status information.

[0090] Specifically, the service instance status monitoring interface is the Metrics interface. The load balancer controller periodically accesses the Metrics interface to synchronize the service instance status information in real time, including availability and capacity information, which is used as the basis for judging service degradation, rate limiting, and circuit breaking, and then adjusting the rule algorithms corresponding to the load balancing strategy.

[0091] In this approach, the accuracy of the rule algorithm execution results is improved by monitoring the service instance status information and adjusting the corresponding rules of the load balancing strategy accordingly. Compared to traditional load balancing tools (such as Nginx), the service discovery mechanism provided in this embodiment is designed based on monitoring metrics. No manual intervention is required when scaling up or down service instances, greatly reducing operation and maintenance costs and change risks. Secondly, because the system can perceive service status and service quality, it will periodically and automatically remove unavailable service instances or degrade low-quality service instances, avoiding multiple occurrences of 50X error codes. Furthermore, compared to ingress-controllers in container scenarios, this solution also achieves cross-cluster traffic forwarding capabilities.

[0092] Furthermore, in step S103, after obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, the following steps are also included:

[0093] Step S1031: If the target address does not exist, execute the second preset rule algorithm and update the target address according to the execution result of the second preset rule algorithm;

[0094] Step S1032: Send the user request to the target service instance based on the updated target address.

[0095] Specifically, the second preset rule algorithm is a rule algorithm pre-set for when the target address does not exist, so that the calculation result of the rule algorithm corresponding to the load balancing strategy is incorrect.

[0096] After obtaining the target address, if the target address exists, forwarding is performed directly; if the target address does not exist, a suitable service instance is calculated according to the second preset rule algorithm, and an attempt is made to calculate the next address. If the second preset rule algorithm is missing, forwarding is performed according to the system default strategy (round-robin).

[0097] In this approach, when the target address does not exist, the target address can be updated by executing a second preset rule algorithm, ensuring that user requests can be correctly sent to the target service instance, thus improving the robustness of the system. Even in the face of changes or unavailability of the target address, the correct routing of requests can be ensured.

[0098] The dynamic load balancing method for gateway services in this embodiment of the invention has the following effects:

[0099] By loading load balancing strategies through custom resource declarations, the corresponding algorithm rules are loaded into a local rule cache. Real-time monitoring of load balancing strategy change events is conducted. When a change event is detected, the algorithm rules are updated in the local rule cache accordingly. This allows for dynamic updates of the algorithm rules for load balancing strategies in each upstream service subset based on real-time monitored change events, thereby adjusting load distribution. Compared to traditional load loading methods, this invention dynamically loads load strategies through custom resource declarations, supporting custom extensions for multiple business scenarios. It allows for customized loading with flexible loading methods, providing different load balancing strategies for different sub-clusters, different services within the same sub-cluster, and the same service within the same sub-cluster at different time periods. The load strategy is broken down from a single large object into multiple fine-grained sub-objects, achieving separation of responsibilities and supporting flexible combination according to scenarios. In other words, based on declarative load strategies, it is flexible to use, simple to manage, easy to maintain, and has lower change costs.

[0100] This invention monitors load balancing policy changes based on events. When a change event is detected, such as deleting, adding, or changing a load balancing policy, the algorithm rules corresponding to the load balancing policy are deleted, added, or replaced in the local rule cache according to the content of the change event. Compared with the method of periodically polling regardless of whether the configuration has changed, this invention is based on event-driven and only performs hot reloading on the configuration that has changed, reducing the overhead caused by system polling.

[0101] By monitoring service instance status information and adjusting the corresponding rule algorithms for load balancing strategies accordingly, the accuracy of rule algorithm execution results is improved. Compared to traditional load balancing tools (such as Nginx), the service discovery mechanism provided in this embodiment is designed based on monitoring metrics. No manual intervention is required when scaling up or down service instances, greatly reducing operational costs and change risks, and improving system stability. Secondly, because the system can perceive service status and service quality, it will periodically and automatically remove unavailable service instances or degrade low-quality service instances, avoiding repeated occurrences of 50X error codes. Furthermore, compared to ingress-controllers in container scenarios, this solution also achieves cross-cluster traffic forwarding capabilities.

[0102] By pre-caching the execution results of the first preset algorithm rule, the target address for the next load is maintained. Under certain circumstances, it can avoid calculating after receiving user requests, thereby improving the system's processing speed and efficiency and reducing customer waiting latency.

[0103] This invention also provides a dynamic load balancing device for gateway services, such as... Figure 7 As shown, it includes:

[0104] The instance set matching module 701 is used to match the corresponding upstream service subset based on the received user request;

[0105] The strategy determination module 702 is used to determine the load balancing strategy based on the matched upstream service subset. The matching relationship between the load balancing strategy and the upstream service subset is pre-stored in the local cache through a custom resource declaration. The algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the change events of the load balancing strategy are monitored in real time. When a change event of the load balancing strategy is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event.

[0106] The strategy execution module 703 is used to obtain the target address by executing the rule algorithm corresponding to the load balancing strategy, and send the user request to the target service instance based on the target address.

[0107] The gateway service dynamic load balancing device of this invention loads load balancing policies through custom resource declarations, loads the corresponding algorithm rules of the load balancing policies into a local rule cache, and monitors load balancing policy change events in real time. When a load balancing policy change event is detected, the algorithm rules corresponding to the load balancing policies are updated in the local rule cache according to the change event. It can dynamically update the algorithm rules corresponding to the load balancing policies in each upstream service subset in the local rule cache according to the real-time monitored change events, thereby adjusting the load distribution. This can solve the problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios and improve load balancing efficiency.

[0108] Furthermore, the load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy;

[0109] Correspondingly, the strategy determination module 702 includes:

[0110] The first strategy loading module is used to match the algorithm rules corresponding to the load balancing strategy from the local algorithm library through a lightweight interpreter when the load balancing strategy is the first type of load balancing strategy.

[0111] The second strategy loading module is used to match the algorithm rules corresponding to the load balancing strategy from external source code and perform rule conversion when the load balancing strategy is the second type of load balancing strategy.

[0112] Furthermore, the execution results obtained from executing the first preset algorithm rule are pre-cached in the local cache. When the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

[0113] Furthermore, the strategy determination module 702 includes:

[0114] The type determination module is used to determine the type of change event;

[0115] The rule change module is used to check whether the currently created or updated policy has been added to the local rule cache if the change event is a creation event or an update event. Then, it performs rule overwriting or rule merging in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing policy. If the change event is a deletion event, it removes the rule algorithm corresponding to the deleted load balancing policy from the local rule cache.

[0116] Furthermore, the gateway service dynamic load balancing device also includes:

[0117] The service monitoring module is used to periodically access the service instance status monitoring interface to obtain service instance status information;

[0118] The algorithm adjustment module is used to adjust the rule algorithm corresponding to the load balancing strategy based on the service instance status information.

[0119] Furthermore, the gateway service dynamic load balancing device also includes:

[0120] The target address update module is used to execute the second preset rule algorithm if the target address does not exist, and update the target address according to the execution result of the second preset rule algorithm.

[0121] The service example sending module is used to send user requests to the target service instance based on the updated target address.

[0122] This invention also provides a schematic diagram of the structure of a computer device, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0123] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0124] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0125] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0126] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0127] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0128] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0129] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0130] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0131] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope of protection.

Claims

1. A dynamic load balancing method for gateway services, characterized in that, include: Match the corresponding upstream service subset based on the received user request; The load balancing strategy is determined based on a matched subset of upstream services. The matching relationship between the load balancing strategy and the upstream service subset is pre-stored in a local cache using a custom resource declaration. The algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the change events of the load balancing strategy are monitored in real time. When a change event is detected, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. The custom resource declaration declares that the type of load balancing strategy includes a predefined method and a runtime loading method. The predefined method mounts the local algorithm library of the load balancing algorithm to a fixed directory. When the load balancing strategy is the first type of load balancing strategy, the lightweight interpreter performs algorithm rule matching from the local algorithm library, retrieves the algorithm file, and then dynamically runs the load algorithm. The runtime loading method, when the load balancing strategy is the second type of load balancing strategy, uses a Configmap as a carrier to import external custom load algorithm source code. The external source code is submitted to the lightweight interpreter as a load balancing plugin for real-time rule conversion. By executing the rule algorithm corresponding to the load balancing strategy, the target address is obtained, and the user request is sent to the target service instance based on the target address.

2. The gateway service dynamic load balancing method according to claim 1, characterized in that, The load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy; Correspondingly, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, including: When the load balancing strategy is a first type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the local algorithm library using a lightweight interpreter; When the load balancing strategy is the second type of load balancing strategy, a lightweight interpreter is used to match the algorithm rules corresponding to the load balancing strategy from the external source code and perform rule conversion.

3. The gateway service dynamic load balancing method according to claim 1, characterized in that, The local cache pre-caches the execution results obtained by executing the first preset algorithm rule. When the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

4. The dynamic load balancing method for gateway services according to claim 1, characterized in that, Based on the change event, update the algorithm rules corresponding to the load balancing strategy in the local rule cache, including: Determine the type of the change event; If the change event is a creation event or an update event, then it is checked whether the currently created or updated strategy has been added to the local rule cache. Then, according to the rule algorithm corresponding to the created or updated load balancing strategy, the rule is overwritten or merged in the local rule cache. If the change event is a deletion event, then the rule algorithm corresponding to the deleted load balancing strategy is removed from the local rule cache.

5. The dynamic load balancing method for gateway services according to claim 1, characterized in that, After sending the user request to the target service instance based on the target address, the process further includes: Periodically access the service instance status monitoring interface to obtain service instance status information; The rule algorithm corresponding to the load balancing strategy is adjusted based on the service instance status information.

6. The dynamic load balancing method for gateway services according to claim 1, characterized in that, After obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, the process further includes: If the target address does not exist, the second preset rule algorithm is executed, and the target address is updated according to the execution result of the second preset rule algorithm; The user request is sent to the target service instance based on the updated target address.

7. A gateway service dynamic load balancing device, characterized in that, include: The instance set matching module is used to match the corresponding upstream service subset based on the received user request. The strategy determination module is used to determine the load balancing strategy based on a matched subset of upstream services. The matching relationship between the load balancing strategy and the upstream service subset is pre-stored in a local cache through a custom resource declaration. The algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the module monitors load balancing strategy change events in real time. When a change event is detected, the module updates the algorithm rules corresponding to the load balancing strategy in the local rule cache based on the change event. The custom resource declaration declares the type of load balancing strategy, including predefined and runtime loading methods. The predefined method mounts the local algorithm library of the load balancing algorithm to a fixed directory. When the load balancing strategy is the first type, the lightweight interpreter matches algorithm rules from the local algorithm library, retrieves algorithm files, and then dynamically runs the load algorithm. Runtime loading, when the load balancing strategy is the second type, uses a Configmap as a carrier to import external custom load algorithm source code. This external source code, as a load balancing plugin, is submitted to the lightweight interpreter for real-time rule conversion. The strategy execution module is used to obtain the target address by executing the rule algorithm corresponding to the load balancing strategy, and send the user request to the target service instance based on the target address.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the gateway service dynamic load balancing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the gateway service dynamic load balancing method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the gateway service dynamic load balancing method as described in any one of claims 1 to 6.

Images