Gateway service dynamic load balancing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of gateway services, and discloses a gateway service dynamic load balancing method, device and equipment and a storage medium, and the method comprises the steps: matching a corresponding upstream service subset according to a received user request; a load balancing strategy is determined based on the matched upstream service subset, the matching relation between the load balancing strategy and the upstream service subset is pre-stored in a local cache through a user-defined resource statement, an algorithm rule corresponding to the load balancing strategy is loaded into a local rule cache, and a change event of the load balancing strategy is monitored in real time; when a change event of the load balancing strategy is monitored, updating an algorithm rule corresponding to the load balancing strategy in a local rule cache according to the change event; and obtaining a target address by executing a rule algorithm corresponding to the load balancing strategy, and sending the user request to the target service instance based on the target address. According to the invention, the load balancing flexibility in a super-large scale cluster scene can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of gateway services, and in particular to a gateway service dynamic load balancing method, device, equipment and storage medium. Background Art

[0002] With the popularization of cloud computing, the deployment and operation of containerized applications have become increasingly important. In this context, Kubernetes (hereinafter referred to as K8s), as an open source container orchestration platform, has become the de facto standard for the deployment and operation of containerized applications.

[0003] In the K8s cluster, the gateway service (kube-gateway) is a gateway service that provides 7-layer load balancing and traffic control, which can greatly reduce the load pressure of the API service. Figure 1 As shown, currently, in the K8s cluster, the gateway service does not support load balancing plug-in capabilities by design. The load logic is integrated through hard coding, which is inefficient and does not support dynamic expansion, resulting in insufficient flexibility in ultra-large-scale cluster scenarios. Summary of the invention

[0004] In view of this, the present invention provides a gateway service dynamic load balancing method, apparatus, device and storage medium to solve the technical problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios.

[0005] In a first aspect, the present invention provides a gateway service dynamic load balancing method, comprising:

[0006] Match the corresponding upstream service subset according to the received user request;

[0007] Determine the load balancing strategy based on the matched upstream service subset, wherein the matching relationship between the load balancing strategy and the upstream service subset is pre-stored in the local cache through the custom resource declaration, load the algorithm rules corresponding to the load balancing strategy into the local rule cache, and monitor the change events of the load balancing strategy in real time. When the change event of the load balancing strategy is monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event;

[0008] By executing the rule algorithm corresponding to the load balancing strategy, the target address is obtained, and the user request is sent to the target service instance based on the target address.

[0009] The gateway service dynamic load balancing method of the embodiment of the present invention loads the load balancing strategy through custom resource declaration, loads the algorithm rules corresponding to the load balancing strategy into the local rule cache, and monitors the change events of the load balancing strategy in real time. When the change event of the load balancing strategy is monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. The algorithm rules corresponding to the load balancing strategy in each upstream service subset can be dynamically updated in the local rule cache according to the change events monitored in real time, and then the load distribution can be adjusted. The problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios can be solved, and the load balancing efficiency can be improved.

[0010] Further, the load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy;

[0011] Correspondingly, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, including:

[0012] When the load balancing strategy is the first type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the local algorithm library through the lightweight interpreter;

[0013] When the load balancing strategy is the second type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the external source code through a lightweight interpreter and the rules are converted.

[0014] In this way, the algorithm rules corresponding to the load balancing strategy are loaded based on a lightweight interpreter, so that the load strategy supports hot expansion. It can not only match the algorithm rules in the local algorithm library, but also match the corresponding rules from the external source code and perform rule conversion, which is convenient for expanding the algorithm content.

[0015] Furthermore, the execution result obtained by executing the first preset algorithm rule is pre-cached in the local cache, and when the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

[0016] In this manner, by pre-caching the execution result of the first preset algorithm rule, in certain circumstances, calculation after receiving a user request can be avoided, thereby improving the processing speed and efficiency of the system and reducing the waiting delay for customers.

[0017] Furthermore, according to the change event, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache, including:

[0018] Determine the type of change event;

[0019] If the change event is a creation event or an update event, it is detected whether the currently created or updated policy has been added to the local rule cache, and then the rules are overwritten or merged in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing policy. If the change event is a deletion event, the rule algorithm corresponding to the deleted load balancing policy is removed from the local rule cache.

[0020] In this way, the local rule cache is dynamically updated according to the change events of the load balancing policy, ensuring the timely update of the load balancing policy. The local rule cache is dynamically updated based on the monitoring service discovery mechanism, without the need for manual intervention and maintenance, reducing operation and maintenance costs and improving system stability.

[0021] Furthermore, after sending the user request to the target service instance based on the target address, the method further includes:

[0022] Periodically access the service instance status monitoring interface to obtain service instance status information;

[0023] Adjust the rule algorithm corresponding to the load balancing strategy based on the service instance status information.

[0024] In this way, the accuracy of the rule algorithm execution results is improved by monitoring the service instance status information and adjusting the rule algorithm corresponding to the load balancing strategy based on the information.

[0025] Furthermore, after obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, the method further includes:

[0026] If the target address does not exist, execute the second preset rule algorithm, and update the target address according to the execution result of the second preset rule algorithm;

[0027] The user request is sent to the target service instance based on the updated target address.

[0028] In this way, when the target address does not exist, the target address can be updated by executing the second preset rule algorithm to ensure that the user request can be correctly sent to the target service instance, thereby improving the robustness of the system and ensuring the correct routing of the request even in the face of changes or unavailability of the target address.

[0029] In a second aspect, the present invention provides a gateway service dynamic load balancing device, comprising:

[0030] An instance set matching module, used to match a corresponding upstream service subset according to a received user request;

[0031] A policy determination module is used to determine a load balancing policy based on a matched subset of upstream services, wherein the matching relationship between the load balancing policy and the subset of upstream services is pre-stored in a local cache through a custom resource declaration, the algorithm rules corresponding to the load balancing policy are loaded into the local rule cache, and the change events of the load balancing policy are monitored in real time. When a change event of the load balancing policy is detected, the algorithm rules corresponding to the load balancing policy are updated in the local rule cache according to the change event;

[0032] The policy execution module is used to obtain the target address by executing the rule algorithm corresponding to the load balancing policy, and send the user request to the target service instance based on the target address.

[0033] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the gateway service dynamic load balancing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the gateway service dynamic load balancing method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0035] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the document processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of a gateway service dynamic load balancing method in the related art;

[0038] Figure 2 It is a flow chart of a gateway service dynamic load balancing method according to an embodiment of the present invention;

[0039] Figure 3 It is a flow chart of another gateway service dynamic load balancing method according to an embodiment of the present invention;

[0040] Figure 4It is a schematic diagram of the process of initializing the dynamic load balancing method of the gateway service according to an embodiment of the present invention;

[0041] Figure 5 It is a schematic diagram of a flow chart of cyclic tuning of a gateway service dynamic load balancing method according to an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of the process of processing user requests by the gateway service dynamic load balancing method according to an embodiment of the present invention;

[0043] Figure 7 is a structural block diagram of a gateway service dynamic load balancing device according to an embodiment of the present invention;

[0044] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] The gateway service dynamic load balancing method of the embodiment of the present invention is applicable to ultra-large-scale clusters, generally with a node scale greater than 5000, and can support a K8S cluster with a container scale of millions through an extended architecture.

[0047] After the gateway service of the K8S cluster receives the user request, it will analyze the situation of the upstream service instance and forward the request to the corresponding API service (kube-apiserver). The API service is the traffic entrance of the K8S cluster, providing access control, authentication, and permission management functions, and accepting client resource query, creation, update, deletion and other requests. The request is connected to the corresponding service instance through the API service to achieve load balancing.

[0048] Based on the related art, the gateway service does not support the load balancing plug-in capability in design, and the load logic is integrated through hard coding (polling), which is inefficient and does not support dynamic expansion, resulting in insufficient flexibility in complex business scenarios and high operation and maintenance costs. An embodiment of the present invention provides a gateway service dynamic load balancing method.

[0049] According to an embodiment of the present invention, an embodiment of a gateway service dynamic load balancing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] In this embodiment, a gateway service dynamic load balancing method is provided, which can be used in a gateway device, and can be specifically applied to a load balancing controller in a gateway device. Figure 3 As shown in the figure, the load balancing controller is the core of the entire load balancing solution and has the following functions: listen to the change events of the load balancing policy and synchronously update the local rule cache; load various custom algorithm rules through the plug-in manager and convert the request address into a proxied service instance address; periodically access the service instance status monitoring interface to synchronize the available status and capacity information of the service instance in real time, which is used to provide a basis for judging service degradation, current limiting and circuit breaking.

[0051] like Figure 2 As shown, the gateway service dynamic load balancing method of the embodiment of the present invention includes the following steps:

[0052] Step S101, matching a corresponding upstream service subset according to a received user request.

[0053] Specifically, the user sends a user request through the client to obtain the corresponding service.

[0054] The user request includes information such as the user domain name and port. When the user request is detected, the upstream service subset (upstream) to which the domain name belongs is matched according to its domain name. The upstream service example set includes multiple service instances (server). After being balanced by the load balancing controller, the user request is sent to one of the service instances for processing.

[0055] Step S102, determine the load balancing strategy based on the matching upstream service subset, wherein the matching relationship between the load balancing strategy and the upstream service subset is pre-stored in the local cache through a custom resource declaration, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the change events of the load balancing strategy are monitored in real time. When the change events of the load balancing strategy are monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change events.

[0056] Specifically, the local cache includes a set of algorithm rule chains sorted by priority. The algorithm rule may be a minimum number of connections, a minimum resource occupation, etc. Each algorithm rule is associated with a processing function.

[0057] Through Custom Resource Definition (CRD), load balancing strategies are configured for each upstream service subset, including the applicable objects of the load balancing strategy (such as different subclusters, different service instances of the same subcluster, and the same service instance of the same subcluster), the effective time, and the corresponding algorithm rules. After the load balancing strategy is configured, each upstream service subset caches the algorithm rules in the corresponding load balancing strategy, and forms a corresponding rule chain through logical combination of multiple algorithm rules. Through custom resource declaration, dynamic load balancing strategies are implemented to support custom extensions for various business scenarios.

[0058] Furthermore, based on event monitoring of change events of load balancing strategies, when change events are detected, such as deletion, addition or change of load balancing strategies, the algorithm rules corresponding to the load balancing strategies are deleted, added or replaced in the local rule cache according to the content of the change event.

[0059] In the related art, load balancing policy rules are obtained by periodic polling (regardless of whether the configuration has changed), which is very performance-intensive in ultra-large-scale scenarios. The present invention is event-driven and only hot-loads the changed configurations, reducing the overhead caused by system polling.

[0060] Step S103, obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, and sending the user request to the target service instance based on the target address.

[0061] Specifically, Figure 6 As shown, since the load balancing controller has converted the load balancing strategy into a rule algorithm and formed a rule chain based on the rule algorithm and stored it in the cache, the target address of the target service instance of the upstream service can be calculated by executing the corresponding rule chain, and the user request can be sent to the target service instance for processing through the target address.

[0062] If no load balancing policy is configured in the upstream service subset to which the domain name belongs, load balancing is performed according to a pre-set policy, such as a round-robin policy.

[0063] The gateway service dynamic load balancing method of the embodiment of the present invention loads the load balancing strategy through custom resource declaration, loads the algorithm rules corresponding to the load balancing strategy into the local rule cache, and monitors the change events of the load balancing strategy in real time. When the change event of the load balancing strategy is monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. The algorithm rules corresponding to the load balancing strategy in each upstream service subset can be dynamically updated in the local rule cache according to the change events monitored in real time, and then the load distribution can be adjusted. The problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios can be solved, and the load balancing efficiency can be improved.

[0064] The gateway service dynamic load balancing method of the embodiment of the present invention implements a dynamic load balancing strategy through custom resource declaration, supports custom extensions for various business scenarios, and is event-driven, which only hot-loads the changed configuration, reducing the overhead caused by system polling.

[0065] In some embodiments, the load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy;

[0066] Correspondingly, in step S102, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, including:

[0067] Step S1021, when the load balancing strategy is the first type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the local algorithm library through a lightweight interpreter.

[0068] Step S1022, when the load balancing strategy is the second type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the external source code through a lightweight interpreter and the rules are converted.

[0069] Specifically, the load balancing controller plug-in manager integrates a lightweight interpreter in the plug-in manager, and the plug-in manager loads the corresponding plug-in according to the configuration or command line parameters.

[0070] The load balancing plug-in and its algorithm library supported by the system are loaded through the plug-in manager, and the rules defined by the load balancing plug-in are converted into a series of intermediate processing flows through a lightweight interpreter. Each algorithm rule is a load balancing plug-in.

[0071] Declare the type of load balancing strategy in the custom resource declaration. Correspondingly, the load balancing plug-in supports two methods: predefined and runtime loading.

[0072] The predefined method is to mount the local algorithm library (i.e. code file) of the load balancing algorithm to a fixed directory. When the load balancing strategy is the first type of load balancing strategy, the lightweight interpreter matches the algorithm rules from the local algorithm library, retrieves the algorithm file, and then dynamically runs the load algorithm through the lightweight interpreter.

[0073] Runtime loading is when the load balancing strategy is the second type of load balancing strategy. Configmap is used as a carrier to import external customized load algorithm source code (ie external source code). The external source code is used as a load balancing plug-in and submitted to the lightweight interpreter for real-time rule conversion.

[0074] Retrieve the algorithm rules supported by the load balancing strategy according to the configured local algorithm library or external algorithm library, and convert them into hook functions and their input parameters. If the algorithm rule does not exist, an error message is returned.

[0075] In this way, the algorithm rules corresponding to the load balancing strategy are loaded based on a lightweight interpreter, so that the load strategy supports hot expansion. It can not only match the algorithm rules in the local algorithm library, but also match the corresponding rules from the external source code and perform rule conversion, which is convenient for expanding the algorithm content.

[0076] In some embodiments, the local cache pre-caches the execution result obtained by executing the first preset algorithm rule, and when the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

[0077] Specifically, the local cache of the load balancing controller is a set of load rule chains sorted by priority, and each rule is associated with a processing function. In order to improve load efficiency, for specific algorithm scenarios, a first preset algorithm rule will be set for it, such as the minimum number of connections, the minimum resource usage, and similar load rules. The next access address is calculated in advance according to the first preset algorithm rule to avoid increasing the client request delay due to the calculation process.

[0078] In this manner, by pre-caching the execution result of the first preset algorithm rule, in certain circumstances, calculation after receiving a user request can be avoided, thereby improving the processing speed and efficiency of the system and reducing the waiting delay for customers.

[0079] In some embodiments, in step S102, according to the change event, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache, including:

[0080] Step S1023, determining the type of the change event;

[0081] Step S1024, if the change event is a creation event or an update event, detect whether the currently created or updated policy has been added to the local rule cache, and then overwrite or merge rules in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing policy. If the change event is a deletion event, remove the rule algorithm corresponding to the deleted load balancing policy from the local rule cache.

[0082] Specifically, the load balancing controller also includes a tuner, which updates the algorithm rules of the local cache by monitoring load balancing policy change events, such as addition time, deletion time and update events.

[0083] like Figure 4As shown in the figure, before the gateway service is deployed, the load balancing policy object is registered in the cluster where the control plane is located. When the gateway service is started, the load balancing controller is loaded. The load balancing controller includes a tuner and a plug-in manager. The tuner updates the locally cached load balancing rules by listening to load balancing policy change events (addition, deletion, update), and the plug-in manager loads the corresponding plug-in according to the configuration or command line parameters.

[0084] like Figure 5 As shown in the figure, when the gateway control plane creates, updates, or deletes a load balancing policy, the load balancing controller automatically enters a loop tuning process. When creating or updating a load balancing policy, the load balancing controller first detects whether the algorithm rules defined by the load balancing policy have been added to the local rule chain. If so, and the merge policy of the load balancing policy is to overwrite, the algorithm rules in the custom resource declaration of the extracted object are directly overwritten in the cache, otherwise the policy is merged. When deleting a load balancing policy, the load balancing controller removes the algorithm rules and their processing functions in the cache.

[0085] In addition, deleting a load balancing policy will trigger the load balancing controller to remove the corresponding algorithm rules. If the corresponding algorithm rules are being referenced, deletion is prohibited.

[0086] In this way, the local rule cache is dynamically updated based on the change events of the monitored load balancing policy, ensuring the timely update of the load balancing policy. The local rule cache is dynamically updated based on the monitored service discovery mechanism, without the need for manual intervention and maintenance, reducing operation and maintenance costs and improving system stability.

[0087] Further, in step S103, after sending the user request to the target service instance based on the target address, the method further includes:

[0088] Step S104, periodically access the service instance status monitoring interface to obtain service instance status information;

[0089] Step S105: adjusting the rule algorithm corresponding to the load balancing strategy based on the service instance status information.

[0090] Specifically, the service instance status monitoring interface is the Metrics interface. The load balancing controller periodically accesses the Metrics interface to synchronize the status information of the service instance in real time, including the available status and capacity information, etc., to provide a basis for judging service degradation, current limiting, and circuit breaking, and then adjust the rule algorithm corresponding to the load balancing strategy.

[0091] In this way, the accuracy of the rule algorithm execution results is improved by monitoring the service instance status information and adjusting the rule algorithm corresponding to the load balancing strategy based on this. Compared with traditional load balancing tools (such as Nginx), the service discovery mechanism provided by the embodiment of the present invention is designed based on monitoring indicators. No human intervention is required when the service instance is expanded or reduced, which greatly reduces the operation and maintenance costs and change risks. Secondly, because the system can perceive the service status and service quality, it will automatically remove unavailable service instances or downgrade low-quality service instances on a regular basis to avoid multiple occurrences of 50X error codes. At the same time, compared with the ingress-controller in the container scenario, this solution also realizes the ability to forward traffic across clusters.

[0092] Furthermore, in step S103, after obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, the method further includes:

[0093] Step S1031, if the target address does not exist, execute the second preset rule algorithm, and update the target address according to the execution result of the second preset rule algorithm;

[0094] Step S1032: Send the user request to the target service instance based on the updated target address.

[0095] Specifically, the second preset rule algorithm is a rule algorithm that is pre-set for when the target address does not exist, so as to avoid the situation where the calculation result of the rule algorithm corresponding to the load balancing strategy is incorrect.

[0096] After obtaining the target address, if the target address exists, forward it directly; if the target address does not exist, calculate the appropriate service instance according to the second preset rule algorithm and try to calculate the next address. If the second preset rule algorithm is missing, forward it according to the system default strategy (polling).

[0097] In this way, when the target address does not exist, the target address can be updated by executing the second preset rule algorithm to ensure that the user request can be correctly sent to the target service instance, thereby improving the robustness of the system and ensuring the correct routing of the request even in the face of changes or unavailability of the target address.

[0098] The gateway service dynamic load balancing method of the embodiment of the present invention has the following effects:

[0099] The load balancing strategy is loaded through custom resource declaration, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the change events of the load balancing strategy are monitored in real time. When the change event of the load balancing strategy is monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. According to the change event monitored in real time, the algorithm rules corresponding to the load balancing strategy in each upstream service subset can be dynamically updated in the local rule cache, thereby adjusting the load distribution. Compared with the traditional load loading form, the load strategy of the present invention dynamically loads the load strategy through custom resource declaration, supports custom extension of multiple business scenarios, can be customized to load, and the loading form is flexible. Different load balancing strategies can be provided for different subclusters, different services of the same subcluster, and the same service of the same subcluster in different time periods. The load strategy is split from a single large object into multiple fine-grained sub-objects to achieve separation of responsibilities and support flexible combination by scenario, that is, based on declarative load strategy, it is flexible to use, simple to manage, convenient to maintain, and has lower change cost.

[0100] Event-based monitoring of change events of load balancing policies. When a change event is detected, such as deletion, addition or change of load balancing policies, the algorithm rules corresponding to the load balancing policies are deleted, added or replaced in the local rule cache according to the content of the change event. Compared with the regular polling method regardless of whether the configuration has changed, the present invention is event-driven and will only hot-load the changed configuration, reducing the overhead caused by system polling.

[0101] By monitoring the service instance status information and adjusting the rule algorithm corresponding to the load balancing strategy based on this, the accuracy of the rule algorithm execution results is improved. Compared with traditional load balancing tools (such as Nginx), the service discovery mechanism provided by the embodiment of the present invention is designed based on monitoring indicators. No human intervention is required when the service instance is expanded or reduced, which greatly reduces the operation and maintenance costs and change risks, and improves system stability. Secondly, because the system can perceive the service status and service quality, it will automatically remove unavailable service instances or downgrade low-quality service instances on a regular basis to avoid multiple occurrences of 50X error codes. At the same time, compared with the ingress-controller in the container scenario, this solution also realizes the ability to forward traffic across clusters.

[0102] By pre-caching the execution results of the first preset algorithm rules, the target address of the next load is maintained. In certain circumstances, calculation after receiving the user request can be avoided, thereby improving the processing speed and efficiency of the system and reducing customer waiting delay.

[0103] The embodiment of the present invention also provides a gateway service dynamic load balancing device, such as Figure 7 As shown, including:

[0104] An instance set matching module 701 is used to match a corresponding upstream service subset according to a received user request;

[0105] A policy determination module 702 is used to determine a load balancing policy based on a matched upstream service subset, wherein the matching relationship between the load balancing policy and the upstream service subset is pre-stored in a local cache through a custom resource declaration, the algorithm rules corresponding to the load balancing policy are loaded into the local rule cache, and the change events of the load balancing policy are monitored in real time. When a change event of the load balancing policy is monitored, the algorithm rules corresponding to the load balancing policy are updated in the local rule cache according to the change event;

[0106] The policy execution module 703 is used to obtain the target address by executing the rule algorithm corresponding to the load balancing policy, and send the user request to the target service instance based on the target address.

[0107] The gateway service dynamic load balancing device of the embodiment of the present invention loads the load balancing strategy through custom resource declaration, loads the algorithm rules corresponding to the load balancing strategy into the local rule cache, and monitors the change events of the load balancing strategy in real time. When the change event of the load balancing strategy is monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event. According to the change events monitored in real time, the algorithm rules corresponding to the load balancing strategy in each upstream service subset can be dynamically updated in the local rule cache, thereby adjusting the load distribution. This can solve the problem of insufficient load balancing flexibility in ultra-large-scale cluster scenarios and improve load balancing efficiency.

[0108] Further, the load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy;

[0109] Correspondingly, the policy determination module 702 includes:

[0110] A first strategy loading module, used for matching the algorithm rules corresponding to the load balancing strategy from the local algorithm library through a lightweight interpreter when the load balancing strategy is a first type of load balancing strategy;

[0111] The second strategy loading module is used to match the algorithm rules corresponding to the load balancing strategy from the external source code through a lightweight interpreter and perform rule conversion when the load balancing strategy is the second type of load balancing strategy.

[0112] Furthermore, the execution result obtained by executing the first preset algorithm rule is pre-cached in the local cache, and when the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

[0113] Further, the policy determination module 702 includes:

[0114] Type judgment module, used to judge the type of change event;

[0115] The rule change module is used to detect whether the currently created or updated policy has been added to the local rule cache if the change event is a creation event or an update event, and then overwrite or merge the rules in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing policy. If the change event is a deletion event, the rule algorithm corresponding to the deleted load balancing policy is removed from the local rule cache.

[0116] Furthermore, the gateway service dynamic load balancing device also includes:

[0117] The service monitoring module is used to periodically access the service instance status monitoring interface to obtain service instance status information;

[0118] The algorithm adjustment module is used to adjust the rule algorithm corresponding to the load balancing strategy based on the service instance status information.

[0119] Furthermore, the gateway service dynamic load balancing device also includes:

[0120] A target address updating module, used for executing a second preset rule algorithm if the target address does not exist, and updating the target address according to the execution result of the second preset rule algorithm;

[0121] The service instance sending module is used to send the user request to the target service instance based on the updated target address.

[0122] The embodiment of the present invention also provides a schematic diagram of the structure of a computer device, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0123] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0124] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0125] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0127] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0128] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0129] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0130] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0131] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope of protection.

Claims

1. A gateway service dynamic load balancing method, characterized in that: include: Match the corresponding upstream service subset according to the received user request; Determine a load balancing strategy based on a matched upstream service subset, wherein the matching relationship between the load balancing strategy and the upstream service subset is pre-stored in a local cache through a custom resource declaration, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, and the change events of the load balancing strategy are monitored in real time. When the change event of the load balancing strategy is monitored, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache according to the change event; The target address is obtained by executing the rule algorithm corresponding to the load balancing strategy, and the user request is sent to the target service instance based on the target address.

2. The gateway service dynamic load balancing method according to claim 1, characterized in that: The load balancing strategy includes a first type of load balancing strategy and a second type of load balancing strategy; Correspondingly, the algorithm rules corresponding to the load balancing strategy are loaded into the local rule cache, including: When the load balancing strategy is a first type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the local algorithm library through a lightweight interpreter; When the load balancing strategy is a second type of load balancing strategy, the algorithm rules corresponding to the load balancing strategy are matched from the external source code through a lightweight interpreter and the rules are converted.

3. The gateway service dynamic load balancing method according to claim 1, characterized in that: The local cache pre-caches an execution result obtained by executing the first preset algorithm rule. When the rule algorithm to be executed is the same as the first preset algorithm rule, the execution result is directly called.

4. The gateway service dynamic load balancing method according to claim 1, characterized in that: According to the change event, the algorithm rules corresponding to the load balancing strategy are updated in the local rule cache, including: Determine the type of the change event; If the change event is a creation event or an update event, it is detected whether the currently created or updated policy has been added to the local rule cache, and then the rules are overwritten or merged in the local rule cache according to the rule algorithm corresponding to the created or updated load balancing policy. If the change event is a deletion event, the rule algorithm corresponding to the deleted load balancing policy is removed from the local rule cache.

5. The gateway service dynamic load balancing method according to claim 1, characterized in that: After sending the user request to the target service instance based on the target address, the method further includes: Periodically access the service instance status monitoring interface to obtain service instance status information; The rule algorithm corresponding to the load balancing strategy is adjusted based on the service instance status information.

6. The gateway service dynamic load balancing method according to claim 1, characterized in that: After obtaining the target address by executing the rule algorithm corresponding to the load balancing strategy, the method further includes: If the target address does not exist, executing a second preset rule algorithm, and updating the target address according to the execution result of the second preset rule algorithm; The user request is sent to the target service instance based on the updated target address.

7. A gateway service dynamic load balancing device, characterized in that: include: An instance set matching module, used to match a corresponding upstream service subset according to a received user request; A policy determination module, used to determine a load balancing policy based on a matched upstream service subset, wherein the matching relationship between the load balancing policy and the upstream service subset is pre-stored in a local cache through a custom resource declaration, the algorithm rules corresponding to the load balancing policy are loaded into the local rule cache, and the change events of the load balancing policy are monitored in real time. When a change event of the load balancing policy is detected, the algorithm rules corresponding to the load balancing policy are updated in the local rule cache according to the change event; The policy execution module is used to obtain a target address by executing a rule algorithm corresponding to the load balancing policy, and send the user request to a target service instance based on the target address.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the gateway service dynamic load balancing method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the gateway service dynamic load balancing method according to any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to make a computer execute the gateway service dynamic load balancing method as described in any one of claims 1 to 6.

Images