Load balancing method for distributed database and database system

Abstract

The invention discloses a load balancing method for a distributed database and a database system.The load balancing method comprises a distributed cluster composed of at least two nodes and an access agent configured in the distributed cluster and further comprises the following steps that in response to monitored state changes of the nodes in the distributed cluster, the access agent is configured in the distributed cluster; updating an access mechanism of the access agent; and in response to a target access request for the distributed cluster, distributing a target node for the target access request according to the access mechanism of the access agent. The problems that extra software and hardware resources and real-time monitoring are needed and an access mechanism needs to be manually changed and maintained to prevent service faults in a mainstream load balancing mode realized in the market at present are at least solved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a load balancing method and a database system for a distributed database. Background Art

[0002] At the current stage, front-end businesses access database clusters through load balancing. The implementation of load balancing is divided into several categories, namely software layer, hardware layer, and database driver layer, but each implementation method has certain limitations.

[0003] The method implemented through the software layer is: using software such as Nginx, LVS and Haproxy to achieve efficient load balancing. These software usually act as reverse proxy servers, receive requests from clients, and then distribute requests to backend database nodes according to the configured load balancing algorithms, such as polling, weighted polling, and minimum number of connections. The advantages are flexible configuration, easy expansion and management, and load balancing strategies can be customized as needed. The disadvantage is that additional hardware and software resources are required to run the load balancing software, which may increase the complexity and maintenance cost of the system. At the same time, cluster changes such as scaling nodes require manual reconfiguration of the software. Another method is: API Gateway, such as API-Gateway service, can also be used as a load balancing solution. It is located between the client and the database, responsible for receiving and processing requests, and routing requests to the appropriate backend service. The advantage is that in addition to load balancing, API Gateway can also provide authentication, current limiting, circuit breaking and other functions to improve the security and stability of the system. The disadvantage is that it also requires additional hardware and software resources, which may increase the complexity and maintenance cost of the system, and manual maintenance is required in the case of cluster changes.

[0004] The method implemented through the software layer is: use hardware load balancers such as F5, A10 and other hardware load balancers, which implement load balancing functions through specialized hardware devices. Hardware load balancers usually have the characteristics of high performance, high stability and low latency. The advantages are stable performance, strong processing power, and suitable for large-scale concurrent scenarios. The disadvantages are high cost and the need for specialized hardware equipment and maintenance teams.

[0005] The method implemented through the driver layer is: through the supporting driver, all existing node configurations of the database that can provide business support are configured through IPS and other methods, and internal rotation is used to achieve load balancing. The advantage is that no additional software and hardware facilities are required, and the cost is controllable. The disadvantage is that in the case of cluster changes, that is, in the scenario of expansion and contraction, the driver IPS and other configurations need to be re-performed, and the newly added / deleted available node information cannot be automatically identified. Summary of the invention

[0006] The present invention provides a load balancing method and database system for a distributed database, which at least solves the problem that the mainstream load balancing method currently implemented on the market requires additional software and hardware resources and real-time monitoring, and requires manual changes to the maintenance access mechanism to prevent business failures.

[0007] A load balancing method for a distributed database includes a distributed cluster consisting of at least two nodes and an access proxy configured inside the distributed cluster, and further includes the following steps:

[0008] In response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy;

[0009] In response to a target access request for the distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

[0010] Optionally, in response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy includes:

[0011] In response to monitoring at least one of node failure, node shrinkage or node expansion in the distributed cluster, the access mechanism of the access proxy is updated.

[0012] Optionally, in response to monitoring at least one of node failure, node shrinkage, or node expansion in the distributed cluster, updating the access mechanism of the access proxy includes:

[0013] When a node failure or a node shrinkage occurs in the distributed cluster, the node with the node failure or the shrunk node is set as unavailable in the access mechanism;

[0014] When it is monitored that a node expansion occurs in the distributed cluster, the expanded node is added to the access mechanism according to the node function and load capacity of the expanded node.

[0015] Optionally, in response to the target access request for the distributed cluster, allocating a target node for the target access request according to the access mechanism of the access proxy includes:

[0016] In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement;

[0017] According to the target requirement and the node function information, at least one node in the distributed cluster that can meet the target requirement is selected;

[0018] Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node after the target access request is allocated meets expectations;

[0019] The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

[0020] Optionally, in response to the target access request for the distributed cluster, allocating a target node for the target access request according to the access mechanism of the access proxy includes:

[0021] In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement;

[0022] According to the target demand, determining a target time required to respond to the target demand;

[0023] According to the target requirement and the node function information, at least one node in the distributed cluster that can meet the target requirement is selected;

[0024] Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node within the target time after the target access request is allocated meets expectations;

[0025] The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

[0026] Optionally, in response to the target access request for the distributed cluster, allocating a target node for the target access request according to the access mechanism of the access proxy includes:

[0027] In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement;

[0028] According to the target demand, determining a target time required to respond to the target demand;

[0029] According to the target demand, target time and node function information, filter out at least one node in the distributed cluster that can meet the target demand and target time;

[0030] Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node within the target time after the target access request is allocated meets expectations;

[0031] The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

[0032] Optionally, the nodes in the distributed cluster are divided into long-time task nodes and short-time task nodes, the long-time task nodes are configured to process demands that require time exceeding a first threshold, and the short-time task nodes are configured to process demands that require time not exceeding the first threshold;

[0033] The step of screening out at least one node in the distributed cluster that can meet the target demand and the target time according to the target demand, the target time, and the node function information includes:

[0034] According to the target time, determining whether the target time exceeds a preset first threshold;

[0035] When the target time exceeds a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is selected from the long-time task nodes according to the target demand and the node function information;

[0036] When the target time does not exceed a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is screened from the short-time task nodes according to the target demand and the node function information.

[0037] Optionally, according to the node load information and the load balancing mechanism, a target node is allocated to the target access request so that the load distribution of each node within the target time after the target access request is allocated meets expectations, including:

[0038] When the target time exceeds a preset first threshold, a target node is allocated to the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each long-duration task node within the target time after the target access request is allocated meets expectations;

[0039] When the target time does not exceed a preset first threshold, a target node is allocated to the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each short-time task node within the target time after the target access request is allocated meets expectations.

[0040] In another aspect, a load balancing database system for a distributed database includes a distributed cluster consisting of at least two nodes and an access proxy and a monitoring component configured within the distributed cluster;

[0041] The monitoring component is configured as follows:

[0042] In response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy;

[0043] The access proxy is configured to:

[0044] storing the access mechanism;

[0045] In response to a target access request for the distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

[0046] Optionally, in response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy includes:

[0047] In response to monitoring at least one of node failure, node shrinkage or node expansion in the distributed cluster, the access mechanism of the access proxy is updated.

[0048] Optionally, in response to the target access request for the distributed cluster, allocating a target node for the target access request according to the access mechanism of the access proxy includes:

[0049] In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement;

[0050] According to the target demand, determining a target time required to respond to the target demand;

[0051] According to the target demand, target time and node function information, filter out at least one node in the distributed cluster that can meet the target demand and target time;

[0052] Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node within the target time after the target access request is allocated meets expectations;

[0053] The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

[0054] On the other hand, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the above method.

[0055] On the other hand, a computer storage medium stores a computer program, and a processor executes the computer program to implement the above method.

[0056] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0057] The present invention provides a load balancing method and database system for a distributed database, including a distributed cluster consisting of at least two nodes and an access agent configured inside the distributed cluster, and further including the following steps: in response to monitoring the state change of the node in the distributed cluster, updating the access mechanism of the access agent; in response to a target access request for the distributed cluster, allocating a target node for the target access request according to the access mechanism of the access agent. At least the problem that the mainstream load balancing method currently implemented on the market requires additional software and hardware resources and real-time monitoring, and requires manual changes to maintain the access mechanism to prevent business failures is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the specific implementation of the present application or the technical solution in the prior art, the following is a brief introduction to the drawings required for the specific implementation or the prior art description. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0059] Figure 1 A schematic diagram of a flow chart of a load balancing method for a distributed database in this application;

[0060] Figure 2 This is a schematic diagram of the structure of a computer device in this application.

[0061] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory.

[0062] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0064] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0065] Example 1

[0066] like Figure 1 As shown, a load balancing method for a distributed database includes a distributed cluster consisting of at least two nodes and an access proxy configured inside the distributed cluster, and further includes the following steps:

[0067] S1. In response to monitoring the status change of a node in a distributed cluster, update the access mechanism of the access proxy.

[0068] Optionally, a state change of a node in a distributed cluster is used to indicate a change in a request response capability of the node in the distributed cluster.

[0069] Optionally, methods for monitoring the status changes of nodes in a distributed cluster include using the monitoring mechanism of Zookeeper, using a distributed database cluster monitoring tool, and monitoring the node status and data changes in the cluster by writing scripts or programs to periodically poll. Specifically, distributed database cluster monitoring tools include Prometheus, Grafana, Zabbix, etc.

[0070] Optionally, the access mechanism of the access proxy may be stored in the access proxy or elsewhere in the distributed cluster.

[0071] S2. In response to a target access request for a distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

[0072] Optionally, the access request may be sent by a user's terminal or by other network devices such as a gateway.

[0073] Optionally, the access request may include URL parameters, path parameters, HTTP request header parameters, and request body parameters.

[0074] Optionally, the access mechanism is configured to allocate a scheme, mapping, mechanism or regular expression, etc. of a target node to the access request based on the received target access request and the function or load condition of each node in the distributed cluster.

[0075] Example 2

[0076] Based on Embodiment 1, this embodiment provides a load balancing method for a distributed database, including a distributed cluster consisting of at least two nodes and an access proxy configured inside the distributed cluster, and further including the following steps:

[0077] S1. In response to monitoring the status change of a node in a distributed cluster, update the access mechanism of the access proxy.

[0078] Optionally, a state change of a node in a distributed cluster is used to indicate a change in a request response capability of the node in the distributed cluster.

[0079] Optionally, methods for monitoring the status changes of nodes in a distributed cluster include using the monitoring mechanism of Zookeeper, using a distributed database cluster monitoring tool, and monitoring the node status and data changes in the cluster by writing scripts or programs to periodically poll. Specifically, distributed database cluster monitoring tools include Prometheus, Grafana, Zabbix, etc.

[0080] Optionally, the access mechanism of the access proxy may be stored in the access proxy or elsewhere in the distributed cluster.

[0081] Optionally, in response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy includes:

[0082] In response to monitoring at least one of node failure, node shrinkage, or node expansion in the distributed cluster, an access mechanism of the access proxy is updated.

[0083] Specifically, node failure, node shrinkage and node expansion of a distributed cluster are operations and concepts in cluster management. Node failure refers to the downtime or "zombie" phenomenon of server nodes that make up the distributed system; node shrinkage refers to removing some nodes from a distributed cluster to reduce the size of the cluster or resource consumption; node expansion refers to adding new nodes to a distributed cluster to increase the size of the cluster or resource processing capacity.

[0084] Optionally, in response to monitoring at least one of node failure, node shrinkage, or node expansion in the distributed cluster, updating the access mechanism of the access proxy includes:

[0085] When a node failure or node shrinkage is detected in a distributed cluster, the node with the node failure or the shrinkage node is set as unavailable in the access mechanism;

[0086] When node expansion is detected in the distributed cluster, the expanded node is added to the access mechanism based on the node function and load capacity of the expanded node.

[0087] Specifically, the shrinking node is a node that needs to be removed when the node shrinks in the distributed cluster; the expanding node is a node that needs to be added when the node expands in the distributed cluster.

[0088] Optionally, setting the node with node failure or the shrunk node to be unavailable in the access mechanism may be by deleting the scheme, mapping, mechanism, regular expression or other data related to the node with node failure or the shrunk node in the access mechanism, or by marking the node with node failure or the shrunk node from available to unavailable in the access mechanism.

[0089] Optionally, the expansion node is added to the access mechanism according to the node function and load capacity of the expansion node, which may be by adding a scheme, mapping, mechanism, regular expression or other data related to the expansion node to the access mechanism, or by marking the expansion node from unavailable to available in the access mechanism.

[0090] S2. In response to a target access request for a distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

[0091] Optionally, the access request may be sent by a user's terminal or by other network devices such as a gateway.

[0092] Optionally, the access request may include URL parameters, path parameters, HTTP request header parameters, and request body parameters.

[0093] Optionally, the access mechanism is configured to allocate a scheme, mapping, mechanism or regular expression, etc. of a target node to the access request based on the received target access request and the function or load condition of each node in the distributed cluster.

[0094] Optionally, in response to a target access request for a distributed cluster, allocating a target node for the target access request according to an access mechanism of an access proxy includes:

[0095] In response to a target access request for a distributed cluster, parsing the target access request to obtain a target requirement;

[0096] According to the target requirements and node function information, at least one node in the distributed cluster that can meet the target requirements is selected;

[0097] According to the node load information and load balancing mechanism, the target node is assigned to the target access request, so that the load distribution of each node after the target access request is assigned meets the expectation;

[0098] The target node belongs to at least one node in the distributed cluster that can meet the target requirements; node function information, node load information and load balancing mechanism belong to the access mechanism.

[0099] Optionally, in response to a target access request for a distributed cluster, in the step of parsing the target access request to obtain a target requirement, the target access request may be an access request such as requesting a service, requesting a function, uploading data, or downloading data. According to the target access request, the distributed cluster may parse the target access request to obtain the operation expected by the target access request, that is, to obtain the target requirement. The target requirement is a requirement, and the requirement may be a specific service, function, operation, or downloading a specific piece of data.

[0100] Optionally, in the step of screening out at least one node in the distributed cluster that can meet the target requirements based on the target requirements and the node function information, the node function information refers to the information about the functions that the node can achieve. The node function information can be stored in the distributed cluster via a data linked list. Through this step, the nodes that meet the target requirements can be quickly screened out.

[0101] Optionally, in the step of allocating target nodes to target access requests based on node load information and load balancing mechanism so that the load distribution of each node after the target access request is allocated meets expectations, the node load information refers to the load condition of the node, which can be 100% or a specific value, such as bandwidth usage. The node load information can be stored in the corresponding node or stored uniformly. The node load information can be actively updated or uploaded by each node or obtained by querying the access agent when needed. The load balancing mechanism refers to the scheme, mapping, mechanism, logic or regular expression, etc., for allocating nodes based on target requirements and the load condition of each node in the distributed cluster so that the load distribution of each node after allocation meets expectations. The load distribution meets expectations, which can be the minimum standard deviation of the load rate of qualified nodes or the minimum standard deviation of the weighted calculated load rate or other indicators used to indicate load balancing.

[0102] Optionally, a load balancing method for a distributed database, which differs from the above solution in that, in response to a target access request for a distributed cluster, a target node is allocated for the target access request according to an access mechanism of an access proxy, including:

[0103] In response to a target access request for a distributed cluster, parsing the target access request to obtain a target requirement;

[0104] According to the target demand, determine the target time required to respond to the target demand;

[0105] According to the target requirements and node function information, at least one node in the distributed cluster that can meet the target requirements is selected;

[0106] According to the node load information and load balancing mechanism, the target node is assigned to the target access request so that the load distribution of each node within the target time after the target access request is assigned meets the expectation;

[0107] The target node belongs to at least one node in the distributed cluster that can meet the target requirements; node function information, node load information and load balancing mechanism belong to the access mechanism.

[0108] Optionally, according to the target demand, the method for determining the target time required to respond to the target demand may be to obtain the target time as the time required to complete the target demand based on the specific content of the target demand according to preset empirical parameters, indicator system, empirical formula or model.

[0109] Optionally, in the step of allocating target nodes to target access requests based on node load information and load balancing mechanism so that the load distribution of each node within the target time after the target access request is allocated meets the expected requirements, the method for calculating the load of each node within the target time may be to calculate the average load of the node within the target time.

[0110] Optionally, a load balancing method for a distributed database, which differs from the above solution in that, in response to a target access request for a distributed cluster, a target node is allocated for the target access request according to an access mechanism of an access proxy, including:

[0111] In response to a target access request for a distributed cluster, parsing the target access request to obtain a target requirement;

[0112] According to the target demand, determine the target time required to respond to the target demand;

[0113] According to the target demand, target time and node function information, at least one node in the distributed cluster that can meet the target demand and target time is selected;

[0114] According to the node load information and load balancing mechanism, the target node is assigned to the target access request so that the load distribution of each node within the target time after the target access request is assigned meets the expectation;

[0115] The target node belongs to at least one node in the distributed cluster that can meet the target requirements; node function information, node load information and load balancing mechanism belong to the access mechanism.

[0116] Optionally, the nodes in the distributed cluster are divided into at least two types of time task nodes, and different time task nodes are configured to process demands whose required time is within different threshold ranges;

[0117] According to the target demand, target time and node function information, at least one node in the distributed cluster that can meet the target demand and target time is selected, including:

[0118] According to the target time, determining a target time threshold range corresponding to the target time;

[0119] According to the target demand and the node function information, at least one node in the distributed cluster that can meet the target demand and the target time is screened out from the time task nodes corresponding to the target time threshold range.

[0120] Through the above solution, different requirements can be effectively managed, which facilitates efficient and orderly management of the load of each node in the distributed cluster.

[0121] Optionally, the average load rate of the time task nodes corresponding to the requirements that require a longer time is generally lower than the load rate of the time task nodes corresponding to the requirements that require a shorter time. When the average load rate of the time task nodes corresponding to the requirements that require a longer time is higher than the load rate of the time task nodes corresponding to the requirements that require a shorter time within a preset period, the proportion of various time task nodes is adjusted, and a part of the time task nodes corresponding to the requirements that require a shorter time is adjusted to the time task nodes corresponding to the requirements that require a longer time.

[0122] Since shorter-term demands occupy nodes for a shorter period of time, demands come in and go out quickly, and the update frequency is higher, nodes that handle this type of demand can be allowed to have a higher load rate.

[0123] Optionally, the nodes in the distributed cluster are divided into long-time task nodes and short-time task nodes, the long-time task nodes are configured to process demands that require time exceeding a first threshold, and the short-time task nodes are configured to process demands that require time not exceeding the first threshold;

[0124] According to the target demand, target time and node function information, at least one node in the distributed cluster that can meet the target demand and target time is selected, including:

[0125] According to the target time, determining whether the target time exceeds a preset first threshold;

[0126] When the target time exceeds a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is selected from the long-time task nodes according to the target demand and the node function information;

[0127] When the target time does not exceed a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is screened out from the short-time task nodes according to the target demand and the node function information.

[0128] Optionally, according to the node load information and the load balancing mechanism, a target node is allocated to the target access request so that the load distribution of each node within the target time after the target access request is allocated meets expectations, including:

[0129] When the target time exceeds a preset first threshold, a target node is allocated to the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each long-duration task node within the target time after the target access request is allocated meets expectations;

[0130] When the target time does not exceed a preset first threshold, a target node is allocated to the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each short-time task node within the target time after the target access request is allocated meets expectations.

[0131] Optionally, the average load rate of the long-duration task nodes is generally lower than the load rate of the short-duration task nodes. When the average load rate of the long-duration task nodes is higher than the load rate of the short-duration task nodes within a preset period, the ratio of the long-duration task nodes to the short-duration task nodes is adjusted, and a part of the short-duration task nodes is adjusted to the long-duration task nodes.

[0132] Optionally, the method of adjusting part of the short-term task nodes to long-term task nodes is:

[0133] Adjust part of the short-time task nodes to long-time task nodes. The load capacity n of these nodes is to meet The minimum value of n, where x is the total load capacity of the short-time task nodes before adjustment, A is the average load rate of the short-time task nodes before adjustment, y is the total load capacity of the long-time task nodes before adjustment, and B is the average load rate of the long-time task nodes before adjustment.

[0134] Optionally, when the load capacity of each node in the distributed cluster is the same, a portion of the short-time task nodes is adjusted to long-time task nodes, and the number of such nodes N is to satisfy The minimum value of N, N is a positive integer, where X is the total number of short-time task nodes before adjustment, A is the average load rate of short-time task nodes before adjustment, Y is the total number of long-time task nodes before adjustment, and B is the average load rate of long-time task nodes before adjustment.

[0135] Example 3

[0136] A load balancing database system for a distributed database, comprising a distributed cluster consisting of at least two nodes and an access proxy and a monitoring component configured inside the distributed cluster;

[0137] The listener component is configured as:

[0138] In response to monitoring the state changes of nodes in the distributed cluster, updating the access mechanism of the access proxy;

[0139] The access proxy is configured as:

[0140] Storage access mechanism;

[0141] In response to a target access request for the distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

[0142] Optionally, a state change of a node in a distributed cluster is used to indicate a change in a request response capability of the node in the distributed cluster.

[0143] Optionally, methods for monitoring the status changes of nodes in a distributed cluster include using the monitoring mechanism of Zookeeper, using a distributed database cluster monitoring tool, and monitoring the node status and data changes in the cluster by writing scripts or programs to periodically poll. Specifically, distributed database cluster monitoring tools include Prometheus, Grafana, Zabbix, etc.

[0144] Optionally, the access mechanism of the access proxy may be stored in the access proxy or elsewhere in the distributed cluster.

[0145] Optionally, in response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy includes:

[0146] In response to monitoring at least one of node failure, node shrinkage, or node expansion in the distributed cluster, an access mechanism of the access proxy is updated.

[0147] Specifically, node failure, node shrinkage and node expansion of a distributed cluster are operations and concepts in cluster management. Node failure refers to the downtime or "zombie" phenomenon of server nodes that make up the distributed system; node shrinkage refers to removing some nodes from a distributed cluster to reduce the size of the cluster or resource consumption; node expansion refers to adding new nodes to a distributed cluster to increase the size of the cluster or resource processing capabilities.

[0148] Optionally, in response to monitoring at least one of node failure, node shrinkage, or node expansion in the distributed cluster, updating the access mechanism of the access proxy includes:

[0149] When a node failure or node shrinkage is detected in a distributed cluster, the node with the node failure or the shrinkage node is set as unavailable in the access mechanism;

[0150] When node expansion is detected in the distributed cluster, the expanded node is added to the access mechanism based on the node function and load capacity of the expanded node.

[0151] Specifically, the shrinking node is a node that needs to be removed when the node shrinks in the distributed cluster; the expanding node is a node that needs to be added when the node expands in the distributed cluster.

[0152] Optionally, setting the node with node failure or the shrunk node to be unavailable in the access mechanism may be by deleting the scheme, mapping, mechanism, regular expression or other data related to the node with node failure or the shrunk node in the access mechanism, or by marking the node with node failure or the shrunk node from available to unavailable in the access mechanism.

[0153] Optionally, the expansion node is added to the access mechanism according to the node function and load capacity of the expansion node, which may be by adding a scheme, mapping, mechanism, regular expression or other data related to the expansion node to the access mechanism, or by marking the expansion node from unavailable to available in the access mechanism.

[0154] Optionally, the access request may be sent by a user's terminal or by other network devices such as a gateway.

[0155] Optionally, the access request may include URL parameters, path parameters, HTTP request header parameters, and request body parameters.

[0156] Optionally, the access mechanism is configured to allocate a scheme, mapping, mechanism or regular expression, etc. of a target node to the access request based on the received target access request and the function or load condition of each node in the distributed cluster.

[0157] Optionally, in response to a target access request for a distributed cluster, allocating a target node for the target access request according to an access mechanism of an access proxy includes:

[0158] In response to a target access request for a distributed cluster, parsing the target access request to obtain a target requirement;

[0159] According to the target requirements and node function information, at least one node in the distributed cluster that can meet the target requirements is selected;

[0160] According to the node load information and load balancing mechanism, the target node is assigned to the target access request, so that the load distribution of each node after the target access request is assigned meets the expectation;

[0161] The target node belongs to at least one node in the distributed cluster that can meet the target requirements; node function information, node load information and load balancing mechanism belong to the access mechanism.

[0162] Optionally, in response to a target access request for a distributed cluster, in the step of parsing the target access request to obtain a target requirement, the target access request may be a request for a service, a request for a function, uploading data, or downloading data, etc. According to the target access request, the distributed cluster may parse the target access request to obtain the operation desired by the target access request, i.e., obtain the target requirement. The target requirement may be a specific service, function, operation, or downloading of a specific data.

[0163] Optionally, in the step of screening out at least one node in the distributed cluster that can meet the target requirements based on the target requirements and the node function information, the node function information refers to the information about the functions that the node can achieve. The node function information can be stored in the distributed cluster via a data linked list. Through this step, the nodes that meet the target requirements can be quickly screened out.

[0164] Optionally, in the step of allocating target nodes to target access requests based on node load information and load balancing mechanism so that the load distribution of each node after the target access request is allocated meets expectations, the node load information refers to the load condition of the node, which can be 100% or a specific value, such as bandwidth usage. The node load information can be stored in the corresponding node or stored uniformly. The node load information can be actively updated or uploaded by each node or obtained by querying the access agent when needed. The load balancing mechanism refers to the scheme, mapping, mechanism, logic or regular expression, etc., for allocating nodes based on target requirements and the load condition of each node in the distributed cluster so that the load distribution of each node after allocation meets expectations. The load distribution meets expectations, which can be the minimum standard deviation of the load rate of qualified nodes or the minimum standard deviation of the weighted calculated load rate or other indicators used to indicate load balancing.

[0165] Optionally, a load balancing database system for a distributed database, which differs from the above solution in that, in response to a target access request for a distributed cluster, a target node is allocated for the target access request according to an access mechanism of an access proxy, including:

[0166] In response to a target access request for a distributed cluster, parsing the target access request to obtain a target requirement;

[0167] According to the target demand, determine the target time required to respond to the target demand;

[0168] According to the target requirements and node function information, at least one node in the distributed cluster that can meet the target requirements is selected;

[0169] According to the node load information and load balancing mechanism, the target node is assigned to the target access request so that the load distribution of each node within the target time after the target access request is assigned meets the expectation;

[0170] The target node belongs to at least one node in the distributed cluster that can meet the target requirements; node function information, node load information and load balancing mechanism belong to the access mechanism.

[0171] Optionally, according to the target demand, the method for determining the target time required to respond to the target demand may be to obtain the target time as the time required to complete the target demand based on the specific content of the target demand according to preset empirical parameters, indicator system, empirical formula or model.

[0172] Optionally, in the step of allocating target nodes to target access requests based on node load information and load balancing mechanism so that the load distribution of each node within the target time after the target access request is allocated meets the expected requirements, the method for calculating the load of each node within the target time may be to calculate the average load of the node within the target time.

[0173] Optionally, a load balancing database system for a distributed database, which differs from the above solution in that, in response to a target access request for a distributed cluster, a target node is allocated for the target access request according to an access mechanism of an access proxy, including:

[0174] In response to a target access request for a distributed cluster, parsing the target access request to obtain a target requirement;

[0175] According to the target demand, determine the target time required to respond to the target demand;

[0176] According to the target demand, target time and node function information, at least one node in the distributed cluster that can meet the target demand and target time is selected;

[0177] According to the node load information and load balancing mechanism, the target node is assigned to the target access request so that the load distribution of each node within the target time after the target access request is assigned meets the expectation;

[0178] The target node belongs to at least one node in the distributed cluster that can meet the target requirements; node function information, node load information and load balancing mechanism belong to the access mechanism.

[0179] Optionally, the nodes in the distributed cluster are divided into at least two types of time task nodes, and different time task nodes are configured to process demands whose required time is within different threshold ranges;

[0180] According to the target demand, target time and node function information, at least one node in the distributed cluster that can meet the target demand and target time is selected, including:

[0181] According to the target time, determining a target time threshold range corresponding to the target time;

[0182] According to the target demand and the node function information, at least one node in the distributed cluster that can meet the target demand and the target time is screened out from the time task nodes corresponding to the target time threshold range.

[0183] Optionally, the nodes in the distributed cluster are divided into long-time task nodes and short-time task nodes, the long-time task nodes are configured to process demands that require time exceeding a first threshold, and the short-time task nodes are configured to process demands that require time not exceeding the first threshold;

[0184] According to the target demand, target time and node function information, at least one node in the distributed cluster that can meet the target demand and target time is selected, including:

[0185] According to the target time, determining whether the target time exceeds a preset first threshold;

[0186] When the target time exceeds a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is selected from the long-time task nodes according to the target demand and the node function information;

[0187] When the target time does not exceed a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is selected from the short-time task nodes according to the target demand and the node function information;

[0188] According to the node load information and load balancing mechanism, the target node is assigned to the target access request so that the load distribution of each node within the target time after the target access request is assigned meets the expectation, including:

[0189] When the target time exceeds a preset first threshold, a target node is allocated to the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each long-duration task node within the target time after the target access request is allocated meets expectations;

[0190] When the target time does not exceed a preset first threshold, a target node is allocated to the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each short-time task node within the target time after the target access request is allocated meets expectations.

[0191] Example 4

[0192] This embodiment provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0193] Specifically, Figure 2 As shown, Figure 2 This is a structural diagram of a computer device in the present application, and the computer device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or it may be a stable non-volatile memory (NVM), such as at least one disk storage; the processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0194] Those skilled in the art will appreciate that Figure 2 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0195] like Figure 2 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and an application program for implementing a load balancing method for a distributed database.

[0196] exist Figure 2In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in the present application can be set in the electronic device, and the electronic device calls the application program stored in the memory 105 for implementing a load balancing method for a distributed database through the processor 101 to implement the above method.

[0197] Example 5

[0198] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement any of the above methods.

[0199] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0200] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0201] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0202] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0203] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a non-volatile storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0205] The above are only preferred embodiments of the present disclosure. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the present disclosure.

Claims

1. A load balancing method for a distributed database, characterized in that: The method includes a distributed cluster consisting of at least two nodes and an access proxy configured in the distributed cluster, and further includes the following steps: In response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy; In response to a target access request for the distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

2. A load balancing method for a distributed database according to claim 1, characterized in that: The updating of the access mechanism of the access proxy in response to monitoring the state change of the node in the distributed cluster includes: In response to monitoring at least one of node failure, node shrinkage or node expansion in the distributed cluster, the access mechanism of the access proxy is updated.

3. A load balancing method for a distributed database according to claim 3, characterized in that: The updating of the access mechanism of the access proxy in response to monitoring at least one of a node failure, a node shrinkage, or a node expansion in the distributed cluster includes: When a node failure or a node shrinkage occurs in the distributed cluster, the node with the node failure or the shrunk node is set as unavailable in the access mechanism; When it is monitored that a node expansion occurs in the distributed cluster, the expanded node is added to the access mechanism according to the node function and load capacity of the expanded node.

4. A load balancing method for a distributed database according to claim 1, characterized in that: The step of responding to a target access request for the distributed cluster and allocating a target node for the target access request according to an access mechanism of the access proxy includes: In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement; According to the target requirement and the node function information, at least one node in the distributed cluster that can meet the target requirement is selected; Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node after the target access request is allocated meets expectations; The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

5. A load balancing method for a distributed database according to claim 1, characterized in that: The step of responding to a target access request for the distributed cluster and allocating a target node for the target access request according to an access mechanism of the access proxy includes: In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement; According to the target demand, determining a target time required to respond to the target demand; According to the target requirement and the node function information, at least one node in the distributed cluster that can meet the target requirement is selected; Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node within the target time after the target access request is allocated meets expectations; The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

6. A load balancing method for a distributed database according to claim 1, characterized in that: The step of responding to a target access request for the distributed cluster and allocating a target node for the target access request according to an access mechanism of the access proxy includes: In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement; According to the target demand, determining a target time required to respond to the target demand; According to the target demand, target time and node function information, filter out at least one node in the distributed cluster that can meet the target demand and target time; Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node within the target time after the target access request is allocated meets expectations; The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

7. A load balancing method for a distributed database according to claim 6, characterized in that: The nodes in the distributed cluster are divided into long-time task nodes and short-time task nodes, the long-time task nodes are configured to process demands that require time exceeding a first threshold, and the short-time task nodes are configured to process demands that require time not exceeding the first threshold; The step of screening out at least one node in the distributed cluster that can meet the target demand and the target time according to the target demand, the target time, and the node function information includes: According to the target time, determining whether the target time exceeds a preset first threshold; When the target time exceeds a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is selected from the long-time task nodes according to the target demand and the node function information; When the target time does not exceed a preset first threshold, at least one node in the distributed cluster that can meet the target demand and the target time is screened from the short-time task nodes according to the target demand and the node function information.

8. A load balancing database system for a distributed database, characterized in that: It includes a distributed cluster consisting of at least two nodes and an access proxy and a listening component configured inside the distributed cluster; The monitoring component is configured as follows: In response to monitoring a state change of a node in the distributed cluster, updating an access mechanism of the access proxy; The access proxy is configured to: storing the access mechanism; In response to a target access request for the distributed cluster, a target node is allocated for the target access request according to an access mechanism of the access proxy.

9. A load balancing database system for a distributed database according to claim 8, characterized in that: The updating of the access mechanism of the access proxy in response to monitoring the state change of the node in the distributed cluster includes: In response to monitoring at least one of node failure, node shrinkage or node expansion in the distributed cluster, the access mechanism of the access proxy is updated.

10. A load balancing database system for a distributed database according to claim 8, characterized in that: The step of responding to a target access request for the distributed cluster and allocating a target node for the target access request according to an access mechanism of the access proxy includes: In response to a target access request for the distributed cluster, parsing the target access request to obtain a target requirement; According to the target demand, determining a target time required to respond to the target demand; According to the target demand, target time and node function information, filter out at least one node in the distributed cluster that can meet the target demand and target time; Allocate a target node for the target access request according to the node load information and the load balancing mechanism, so that the load distribution of each node within the target time after the target access request is allocated meets expectations; The target node belongs to at least one node in the distributed cluster that can meet the target requirement; the node function information, the node load information and the load balancing mechanism belong to the access mechanism.

Images