A load balancing weight configuration method, device and medium of a database cluster

Abstract

This invention discloses a load balancing weight configuration method, device, and medium for a database cluster, belonging to the field of database read-write separation technology. It addresses the technical problem that existing read-write separation database clusters use a single fixed weight for load balancing, where node weights cannot be synchronized with roles after a master-slave switch, leading to excessive load on the new master node and unbalanced cluster load. The method includes: configuring a weight combination for each node in the database cluster; wherein the weight combination includes at least a master weight and a slave weight; real-time monitoring of the master-slave status of each node; wherein the master-slave status is either a master node or a slave node; dynamically selecting the corresponding weight from the weight combination based on the master-slave status of each node; and distributing front-end requests to each node proportionally according to the selected weight; wherein write requests are distributed to the master node by default.

Description

Technical Field

[0001] This invention relates to the field of database read / write separation technology, and in particular to a method, device and medium for configuring load balancing weights in a database cluster. Background Technology

[0002] In high-concurrency and high-availability database applications, read-write separation architecture is widely adopted. It effectively improves the system's concurrent processing capabilities and data reliability by separating and routing read and write requests to the backend database cluster. A database cluster typically consists of a master node and a standby node. The master node handles write requests and some read requests, while the standby node primarily handles read requests. A load balancing strategy is used to distribute traffic reasonably between the master and standby nodes. Existing read-write separation load balancing implementations often configure a single fixed weight for each node in the cluster, distributing read requests according to the weight ratio. Furthermore, to balance the write load on the master node, the standby node's weight is usually higher than the master node's during initial configuration. Database clusters have a master-slave failover mechanism. When the master node fails or is under maintenance, the standby node can be promoted to the new master node, and the original master node is demoted to a standby node, ensuring business continuity.

[0003] However, existing technologies have significant drawbacks. Node weights are single, fixed values, unbound to primary or standby roles. When a node switches over, its role changes, and the weight configuration cannot be synchronously adapted to the new role. Furthermore, the new primary node retains its original high weight, and with the added write request load, it is prone to overload and increased response latency. Conversely, the original primary node, after becoming a standby node, continues to use its original low weight, resulting in insufficient load utilization and ultimately causing overall cluster load imbalance, reducing system stability and business continuity. Summary of the Invention

[0004] This invention provides a method, device, and medium for configuring load balancing weights in a database cluster, which addresses the following technical problem: Existing read-write separation database clusters use a single fixed weight to achieve load balancing. After a master-slave switch, the node weights cannot be adjusted synchronously with the roles, resulting in excessive load on the new master node and unbalanced cluster load, affecting the stability and performance of the cluster operation.

[0005] The embodiments of the present invention adopt the following technical solutions: On one hand, embodiments of the present invention provide a load balancing weight configuration method for a database cluster, the method comprising: configuring a weight combination for each node in the database cluster; wherein the weight combination includes at least a primary weight and a backup weight; Real-time monitoring of the primary / backup status of each node; wherein, the primary / backup status refers to the primary node or the backup node; Based on the primary / standby status of each node, the corresponding weight is dynamically selected from the weight combination; Based on the selected weights, frontend requests are distributed to each node proportionally; write requests are distributed to the master node by default.

[0006] In one feasible implementation, a weight combination is configured for each node in the database cluster, specifically including: A configuration interface is provided in the database cluster, through which a sovereign weight and a backup weight are configured for each node in the database cluster, and stored as a weight combination in the weight configuration module; The configuration data of the configuration interface is stored in a persistent storage method to ensure that the weight configuration is not lost after the read-write separation tool restarts; the configuration interface supports batch configuration and single node configuration, and also supports modification of weight values.

[0007] In one feasible implementation, the master / standby status of each node is monitored in real time, specifically including: When no master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through active detection. When a master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through passive notification. Based on the obtained status information of each node, it is determined whether each node is currently a primary or backup device, thus obtaining the primary / backup status.

[0008] In one feasible implementation, when no master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through active detection, specifically including: When no master-slave switchover event occurs in the database cluster, the database cluster status query command is called periodically to read the status information of each node in the database cluster; wherein, the status information includes the node status of the current node, that is, whether it is the master node or the slave node.

[0009] In one feasible implementation, when a master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through passive notification, specifically including: Listen for master-slave failover events in the database cluster; When a master-slave failover event is detected, receive the status change message sent by the database cluster; Based on the status change message, determine the current node status of the node that has changed, i.e., whether it is a primary node or a backup node.

[0010] In one feasible implementation, based on the primary / standby status of each node, a corresponding weight is dynamically selected from the weight combination, specifically including: Based on the primary / standby status of each node, the corresponding weight value is quickly matched in the weight combination of each node; Cache the primary / standby status and corresponding weight of each node in memory; When the primary / standby status of a node remains unchanged, the corresponding weight is read directly from the cache, eliminating the need to repeatedly query the configuration. When the primary / backup status of a node changes, the corresponding weight in the cache is updated, and the new weight data is passed to the request distribution module.

[0011] In one feasible implementation, frontend requests are distributed to each node proportionally according to the selected weights, specifically including: Calculate the total weight of all nodes in the database cluster, and calculate the proportion of each node's weight to the total weight; Each node is allocated a corresponding front-end request quota according to the aforementioned ratio; During the round-robin allocation process, front-end requests are allocated to each node according to the aforementioned front-end request quota.

[0012] In one feasible implementation, after distributing frontend requests to nodes proportionally according to selected weights, the method further includes: The system monitors the load usage data of each node in real time. If the load usage data of a node exceeds a preset threshold, its weight is temporarily adjusted to optimize the load balancing effect.

[0013] On the other hand, embodiments of the present invention also provide a load balancing weight configuration device for a database cluster, the device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to execute the load balancing weight configuration method for a database cluster.

[0014] Finally, this embodiment of the invention also provides a storage medium, which is a non-volatile computer-readable storage medium storing at least one program, each program including instructions, which, when executed by a terminal, cause the terminal to execute the aforementioned load balancing weight configuration method for a database cluster.

[0015] Compared with the prior art, the load balancing weight configuration method, device and medium for a database cluster provided in this embodiment of the invention have the following beneficial effects: This invention abandons the single-weight configuration method for database nodes in existing technologies. Instead, it configures two weight values ​​for each backend database node, corresponding to the primary weight when the node is the master node and the backup weight when it is the backup node. The read / write separation tool adds a node master / backup status detection module and a dynamic weight selection module. It obtains the master / backup status of each node in real time and automatically selects the corresponding weight based on the current master / backup status. The master node status uses the primary weight, and the backup node status uses the backup weight. Requests are then distributed according to the selected weight ratio, accurately and quickly achieving load balancing after master / backup failover. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a load balancing weight configuration method for a database cluster is provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a load balancing weight configuration device for a database cluster, provided in an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0018] This invention provides a method for configuring load balancing weights in a database cluster, such as... Figure 1 As shown, the method for configuring load balancing weights in a database cluster specifically includes steps S101-S106: S101. Configure a weight combination for each node in the database cluster; wherein, the weight combination includes at least the sovereign weight and the backup weight.

[0019] Specifically, a configuration interface is provided in the database cluster. Through the configuration interface, a sovereign weight and a backup weight are configured for each node in the database cluster, and stored as a weight combination in the weight configuration module.

[0020] The configuration data of the configuration interface is stored in a persistent storage method to ensure that the weight configuration is not lost after the read-write separation tool restarts; the configuration interface supports batch configuration and single node configuration, and also supports modification of weight values.

[0021] As a feasible implementation method, a weight configuration module is used to configure two weight values ​​for each node in the database cluster: the primary weight (used when the node is the primary node) and the backup weight (used when the node is the backup node). For example, the primary weights of nodes n1, n2, and n3 are configured as 1, 1, and 1, respectively, and the backup weights are configured as 10, 10, and 10, respectively, ensuring that no matter which node is the primary node, its weight is lower than that of the other backup nodes.

[0022] S102. Real-time detection of the primary / backup status of each node; where the primary / backup status is the primary node or the backup node.

[0023] Specifically, when no master-slave switchover event occurs in the database cluster, the status information of each node in the database cluster is obtained through active detection. This includes periodically calling the database cluster status query command to read the status information of each node in the database cluster when no master-slave switchover event occurs. The status information includes the current node status, i.e., whether it is a master node or a slave node.

[0024] Furthermore, when a primary / standby switchover event occurs in the database cluster, the status information of each node in the database cluster is obtained through passive notification. Specifically, this includes: listening for the primary / standby switchover event in the database cluster; receiving the status change message sent by the database cluster when the primary / standby switchover event is detected; and determining the current node status of the node that has changed, i.e., whether it is a primary node or a standby node, based on the status change message.

[0025] Finally, based on the obtained status information of each node, it is determined whether each node is currently the primary or backup device, thus obtaining the primary / backup status.

[0026] As a feasible implementation method, the status detection module in the read / write separation tool establishes real-time communication with the backend database cluster to obtain the current master / slave status (master node / slave node) of each node. The detection frequency can be configured according to actual business needs, such as checking once every 100ms, to ensure that changes in node status are detected in a timely manner. The status detection can be implemented using the database cluster's built-in status query interface to read the node's role identifier, without the need for additional complex detection logic development.

[0027] S103. Based on the primary / standby status of each node, dynamically select the corresponding weight in the weight combination.

[0028] Specifically, based on the primary / standby status of each node, the corresponding weight value is quickly matched from the weight combinations of each node. The primary / standby status and corresponding weight of each node are cached in memory.

[0029] When the primary / standby status of a node remains unchanged, the corresponding weight is read directly from the cache without needing to query the configuration repeatedly. When the primary / standby status of a node changes, the corresponding weight in the cache is updated, and the new weight data is passed to the request distribution module.

[0030] As a feasible implementation method, the weight selection module receives the primary / standby status of each node from the status detection module, and combines it with the primary / standby weight data in the weight configuration module to select the weight corresponding to the current status of each node. Specifically, if a node is currently in primary node status, its primary weight is selected; if a node is currently in standby node status, its standby weight is selected. For example, initially, n1 is the primary node, and n2 and n3 are standby nodes. The weight selection module selects a primary weight of 1 for n1 and a standby weight of 10 for n2 and n3. After a primary / standby switchover, n2 becomes the primary node, and n1 and n3 become standby nodes. The weight selection module then selects a primary weight of 1 for n2 and a standby weight of 10 for n1 and n3.

[0031] S104. Distribute front-end requests to each node proportionally according to the selected weight; write requests are distributed to the master node by default.

[0032] Specifically, the total weight of all nodes in the database cluster is calculated, and the proportion of each node's weight to the total weight is also calculated. Based on this proportion, a corresponding frontend request quota is allocated to each node. During the round-robin allocation process, frontend requests are distributed to each node according to their respective frontend request quotas.

[0033] The system monitors the load usage data of each node in real time. If the load usage data of a node exceeds a preset threshold, its weight is temporarily adjusted to optimize the load balancing effect.

[0034] As a feasible implementation, the request distribution module receives requests initiated by the frontend (mainly read requests; write requests are distributed to the primary node by default). Based on the current weights of each node determined by the weight selection module, it calculates the proportion of requests each node should receive and distributes the requests to the corresponding nodes according to this proportion, thus achieving load balancing. For example, before and after a primary / standby switchover, the request distribution ratio is 1:10:10 (i.e., the primary node has a weight of 1, and the two standby nodes each have a weight of 10), ensuring that the load on the primary node is always lower than that on the standby nodes, and that the load on each node remains balanced.

[0035] The detailed working process of each module provided by this invention is as follows: Weight Configuration Module: This module stores the primary and backup weights of each database node, provides a configuration interface, and allows users to adjust the weight values ​​according to actual business scenarios (such as node performance and business volume). The configuration data is persistently stored to ensure that the weight configuration is not lost after the read-write separation tool restarts. The configuration interface supports batch configuration and single node configuration, is easy to operate, and is adaptable to database clusters of different sizes.

[0036] Status Detection Module: This module uses a combination of active detection and passive notification to obtain the primary / standby status of nodes. Active detection reads the node's role information by periodically calling database cluster status query commands, such as the pg_is_in_recovery command in PostgreSQL. Passive notification listens for primary / standby failover events in the database cluster and receives status change messages sent by the cluster when a failover occurs. The two methods complement each other to ensure the real-time performance and accuracy of status detection and avoid untimely weight switching due to detection delays.

[0037] Weight Selection Module: This module has built-in state-weight mapping logic. After receiving state data from the state detection module, it quickly matches the corresponding weight value. To improve response speed, a caching mechanism is adopted to cache the current state and corresponding weight of each node in memory. When the state has not changed, the weight is read directly from the cache without repeated configuration queries. When the state changes, the cache is updated immediately and the new weight data is passed to the request distribution module.

[0038] Request Distribution Module: This module employs a weighted round-robin algorithm to distribute requests based on the current weight ratio of each node. The core logic of the algorithm is to calculate the sum of the weights of all nodes, allocate a request quota corresponding to each weight to each node, and distribute requests according to the quota during the round-robin process to ensure that the request distribution ratio is consistent with the weight ratio. At the same time, this module has a load monitoring function, which can monitor the load status of each node in real time (such as CPU utilization and memory usage). If the load of a node is too high, its weight can be temporarily adjusted, such as by reducing the weight, to further optimize the load balancing effect.

[0039] As a feasible implementation, in the initial state, i.e., time t0, n1 is the master node and n2 and n3 are the backup nodes in the database cluster. The status detection module detects the status of each node, the weight selection module selects a master weight of 1 for n1, and a backup weight of 10 for n2 and n3; the request distribution module distributes requests in a ratio of 1:10:10, and the load is balanced among the nodes.

[0040] When the master-slave switchover is triggered, i.e. at time t1, the database cluster undergoes a master-slave switchover, with n2 becoming the master node and n1 and n3 becoming the slave nodes; the database cluster sends a status change notification to the status detection module.

[0041] During the state update, i.e. at time t2, the state detection module receives the state change notification, re-detects the state of each node, confirms that n2 is the master node and n1 and n3 are the backup nodes, and transmits the updated state information to the weight selection module.

[0042] During the weight switching, i.e. at time t3, the weight selection module switches n2 to primary weight 1 and n1 and n3 to backup weight 10 based on the updated node status, and transmits the updated weight information to the request distribution module.

[0043] Finally, during the request distribution update, i.e. at time t4, the request distribution module receives the updated weight information, maintains the request distribution ratio of 1:10:10, and continues to distribute requests. At this time, n2, as the master node, receives the same number of requests as the original master node n1, so the load will not be too high and the load of each node remains balanced.

[0044] The following example, a database cluster containing three nodes (n1, n2, n3), illustrates the application process of this invention in detail: 1. Weight Configuration: Through the weight configuration module, configure the main weights of n1, n2, and n3 as 1, 1, and 1 respectively, and the backup weights as 10, 10, and 10 respectively; 2. Initial state operation: n1 is the master node, and n2 and n3 are backup nodes; after the state detection module detects this state, the weight selection module selects a master weight of 1 for n1 and a backup weight of 10 for n2 and n3; the request distribution module distributes requests in a ratio of 1:10:10, sending 1 request to n1 and 10 requests to n2 and n3 respectively. As the master node, after the write requests are added, the load of n1 is basically the same as that of n2 and n3. 3. Master-slave switchover: If n1 fails, the database cluster triggers a master-slave switchover, with n2 becoming the master node and n1 and n3 becoming the slave nodes. 4. Weight Switching: The state detection module detects node state changes in a timely manner and transmits the information to the weight selection module; the weight selection module switches n2 to the primary weight 1, and switches n1 and n3 to the backup weight 10. 5. Operation after switchover: The request distribution module still distributes requests in a 1:10:10 ratio. n2, as the master node, receives the same number of requests as the original master node n1, and the load will not be too high. n1 and n3, as backup nodes, receive a normal number of requests, and the load of the three nodes remains balanced. 6. Fault Recovery: After the n1 fault is repaired, if the database cluster switches back to n1 as the master node and n2 and n3 as the backup nodes, the weight selection module will automatically switch n1 to master weight 1 and n2 and n3 to backup weight 10. The request distribution ratio remains unchanged and the load is always balanced.

[0045] In addition, embodiments of the present invention also provide a load balancing weight configuration device for a database cluster, such as... Figure 2 As shown, the load balancing weight configuration devices for the database cluster specifically include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, so that at least one processor can perform the following: Configure a weight combination for each node in the database cluster; wherein the weight combination includes at least a sovereign weight and a backup weight; Real-time monitoring of the primary / backup status of each node; wherein, the primary / backup status refers to the primary node or the backup node; Based on the primary / standby status of each node, the corresponding weight is dynamically selected from the weight combination; Based on the selected weights, frontend requests are distributed to each node proportionally; write requests are distributed to the master node by default.

[0046] Finally, this embodiment of the invention also provides a storage medium, which is a non-volatile computer-readable storage medium storing at least one program, each program including instructions, which, when executed by a terminal, cause the terminal to perform: Configure a weight combination for each node in the database cluster; wherein the weight combination includes at least a sovereign weight and a backup weight; Real-time monitoring of the primary / backup status of each node; wherein, the primary / backup status refers to the primary node or the backup node; Based on the primary / standby status of each node, the corresponding weight is dynamically selected from the weight combination; Based on the selected weights, frontend requests are distributed to each node proportionally; write requests are distributed to the master node by default.

[0047] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0048] The foregoing has described specific embodiments of the present invention. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0049] The above description is merely an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for configuring load balancing weights in a database cluster, characterized in that, The method includes: Configure a weight combination for each node in the database cluster; wherein the weight combination includes at least a sovereign weight and a backup weight; Real-time monitoring of the primary / backup status of each node; wherein, the primary / backup status refers to the primary node or the backup node; Based on the primary / standby status of each node, the corresponding weight is dynamically selected from the weight combination; Based on the selected weights, frontend requests are distributed to each node proportionally; write requests are distributed to the master node by default.

2. The method for configuring load balancing weights in a database cluster according to claim 1, characterized in that, Configure a weight combination for each node in the database cluster, specifically including: A configuration interface is provided in the database cluster, through which a sovereign weight and a backup weight are configured for each node in the database cluster, and stored as a weight combination in the weight configuration module; The configuration data of the configuration interface is stored in a persistent storage method to ensure that the weight configuration is not lost after the read-write separation tool restarts; the configuration interface supports batch configuration and single node configuration, and also supports modification of weight values.

3. The method for configuring load balancing weights in a database cluster according to claim 1, characterized in that, Real-time monitoring of the primary and backup status of each node, specifically including: When no master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through active detection. When a master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through passive notification. Based on the obtained status information of each node, it is determined whether each node is currently a primary or backup device, thus obtaining the primary / backup status.

4. The method for configuring load balancing weights in a database cluster according to claim 3, characterized in that, When no master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through active detection, specifically including: When no master-slave switchover event occurs in the database cluster, the database cluster status query command is called periodically to read the status information of each node in the database cluster; wherein, the status information includes the node status of the current node, that is, whether it is the master node or the slave node.

5. The method for configuring load balancing weights in a database cluster according to claim 3, characterized in that, When a master-slave failover event occurs in the database cluster, the status information of each node in the database cluster is obtained through passive notification, specifically including: Listen for master-slave failover events in the database cluster; When a master-slave failover event is detected, receive the status change message sent by the database cluster; Based on the status change message, determine the current node status of the node that has changed, i.e., whether it is a primary node or a backup node.

6. The load balancing weight configuration method for a database cluster according to claim 1, characterized in that, Based on the primary / standby status of each node, the corresponding weight is dynamically selected from the weight combination, specifically including: Based on the primary / standby status of each node, the corresponding weight value is quickly matched in the weight combination of each node; Cache the primary / standby status and corresponding weight of each node in memory; When the primary / standby status of a node remains unchanged, the corresponding weight is read directly from the cache, eliminating the need to repeatedly query the configuration. When the primary / backup status of a node changes, the corresponding weight in the cache is updated, and the new weight data is passed to the request distribution module.

7. The method for configuring load balancing weights in a database cluster according to claim 1, characterized in that, Based on the selected weights, frontend requests are distributed to each node proportionally, specifically including: Calculate the total weight of all nodes in the database cluster, and calculate the proportion of each node's weight to the total weight; Each node is allocated a corresponding front-end request quota according to the aforementioned ratio; During the round-robin allocation process, front-end requests are allocated to each node according to the aforementioned front-end request quota.

8. The method for configuring load balancing weights in a database cluster according to claim 7, characterized in that, After distributing frontend requests to each node proportionally according to the selected weights, the method further includes: The system monitors the load usage data of each node in real time. If the load usage data of a node exceeds a preset threshold, its weight is temporarily adjusted to optimize the load balancing effect.

9. A load balancing weight configuration device for a database cluster, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to execute a load balancing weight configuration method for a database cluster according to any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium is a non-volatile computer-readable storage medium that stores at least one program, each program including instructions that, when executed by a terminal, cause the terminal to perform a load balancing weight configuration method for a database cluster according to any one of claims 1-8.

Images