Load balancing method and apparatus based on database cluster, and device and medium

Abstract

Provided in the embodiments of the present disclosure are a load balancing method and apparatus based on a database cluster, and a device and a medium. The method comprises: receiving access requests for a database cluster, allocating the access requests to a first database node and a second database node, and migrating, to the first database node, all write requests among first access requests allocated to the second database node; and acquiring the current server resource usage conditions corresponding to the first database node, and on the basis of the current server resource usage conditions, migrating, to the second database node, at least some read requests among second access requests allocated to the first database node. In the embodiments of the present disclosure, read requests of a first database node are migrated to a second database node on the basis of the current server resource usage conditions, such that the load of the first database node is reduced, and the resource usage rate of the second database node is improved, thereby achieving the load balancing of read requests in a database cluster.

Description

Database cluster-based load balancing method, device, equipment and medium

[0001] Priority information

[0002] The present disclosure claims priority to the Chinese patent application No. 202411344418.4, filed on September 25, 2024, and entitled "Database cluster-based load balancing method, device, equipment and medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the field of database load balancing, and particularly relates to a database cluster-based load balancing method, device, equipment and medium. BACKGROUND

[0004] At present, in order to improve the reliability, scalability and performance of the database cluster in the cloud environment, a master-slave dual cloud architecture is generally built, in which the master database is responsible for processing read and write operations, and the slave database is responsible for processing read operations. In this way, the master database only needs to process write operations, and the written data is automatically synchronized to the slave database.

[0005] However, in the case of using this method, when there are many write operations in the business, the database responsible for writing will have a high load, blocking read and write and affecting business requests, and the read database may have idle resources due to fewer read requests. SUMMARY

[0006] In view of the above problems, a database cluster-based load balancing method, device, equipment and medium are provided to overcome the above problems or at least partially solve the above problems, comprising:

[0007] A database cluster-based load balancing method, the database cluster comprising a first database node and a second database node, the first database node being configured to process write requests and read requests, and the second database node being configured to process read requests, the method comprising:

[0008] receiving an access request for the database cluster, distributing the access request to the first database node and the second database node, and migrating all write requests in the first access request distributed to the second database node to the first database node;

[0009] obtaining the current server resource usage corresponding to the first database node, and migrating at least part of the read requests in the second access request distributed to the first database node to the second database node according to the current server resource usage.

[0010] Optionally, the migrating, according to the current server resource usage, at least part of read requests in the second access requests assigned to the first database node to the second database node comprises:

[0011] The read request assignment ratio of the first database node and the second database node is modified according to the current server resource usage, and read requests in the second access requests assigned to the first database node are migrated to the second database node according to the modified read request assignment ratio.

[0012] Optionally, the current server resource usage comprises a current server resource usage rate, and the modifying, according to the current server resource usage, the read request assignment ratio of the first database node and the second database node comprises:

[0013] The current access request frequency of the first database node is obtained.

[0014] The read request assignment ratio of the first database node and the second database node is modified according to the current server resource usage rate and the current access request frequency.

[0015] Optionally, the modifying, according to the current server resource usage rate and the current access request frequency, the read request assignment ratio of the first database node and the second database node comprises:

[0016] The test result of the first database node is obtained, wherein the test result comprises a corresponding relationship between server resource usage and database read-write frequency;

[0017] The read request information to be migrated is determined according to the current server resource usage rate, the current access request frequency, and the corresponding relationship between server resource usage and database read-write frequency.

[0018] The read request assignment ratio of the first database node and the second database node is modified according to the read request information to be migrated.

[0019] Optionally, the current server resource usage comprises a current server resource usage rate.

[0020] Before the modifying, according to the current server resource usage, the read request assignment ratio of the first database node and the second database node, the method further comprises:

[0021] It is judged whether the current server resource usage rate is greater than a preset server resource usage rate.

[0022] When the current server resource usage is greater than the preset server resource usage, the read request distribution ratio of the first database node and the second database node is modified according to the current server resource usage.

[0023] Optionally, the assigning the access request to the first database node and the second database node comprises:

[0024] The access request is assigned to the first database node and the second database node according to the currently configured access request distribution ratio.

[0025] Optionally, the method further comprises:

[0026] The currently configured access request distribution ratio is modified in response to a ratio modification request.

[0027] Optionally, after the assigning the access request to the first database node and the second database node, the method further comprises:

[0028] The read request and the write request in the assigned access request are separated in the first database node and the second database node.

[0029] Optionally, the method further comprises:

[0030] The data in the first database node is synchronized to the second database node.

[0031] Optionally, the first database node and the second database node are database nodes in different cloud platforms.

[0032] A load balancing device based on a database cluster, the database cluster comprising a first database node and a second database node, the first database node being configured to process write requests and read requests, and the second database node being configured to process read requests, the device comprising:

[0033] An access request assigning module configured to receive an access request for the database cluster, assign the access request to the first database node and the second database node, and migrate all write requests in the first access request assigned to the second database node to the first database node.

[0034] A read request migration module configured to obtain a current server resource usage corresponding to the first database node, and migrate at least part of read requests in the second access request assigned to the first database node to the second database node according to the current server resource usage.

[0035] An electronic device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the method as described above.

[0036] A computer-readable storage medium having stored thereon a computer program, the computer program, when executed by a processor, implementing the method as described above.

[0037] The embodiments of the present disclosure have the following advantages:

[0038] In the embodiments of the present disclosure, by receiving an access request for a database cluster, assigning the access request to a first database node and a second database node, and migrating all write requests in the first access request assigned to the second database node to the first database node, the current server resource usage corresponding to the first database node is obtained, and at least part of the read requests in the second access request assigned to the first database node are migrated to the second database node according to the current server resource usage, so that the read requests of the first database node are migrated to the second database node according to the current server resource usage, the load of the first database node is reduced, the resource usage rate of the second database node is improved, and the load balancing of the read requests in the database cluster is realized. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the present disclosure, the following will briefly introduce the drawings needed to be used in the description of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0040] FIG. 1 is a schematic diagram of a system architecture in the related art;

[0041] FIG. 2 is a schematic diagram of a system architecture provided by an embodiment of the present disclosure;

[0042] FIG. 3 is a step flowchart of a database cluster-based load balancing method provided by an embodiment of the present disclosure;

[0043] FIG. 4 is a step flowchart of another database cluster-based load balancing method provided by an embodiment of the present disclosure;

[0044] FIG. 5 is a structural block diagram of a database cluster-based load balancing apparatus provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0045] In order to make the above objectives, features and advantages of the present disclosure more obvious and easy to understand, the present disclosure will be further described in detail below in combination with the drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure.

[0046] In the related art, as shown in FIG. 1, the database service can be deployed on the cloud servers of two cloud service operators, A cloud server and B cloud server respectively. Due to the server cost and other reasons, the number of two cloud servers purchased is different. Taking the example that the server resources of A cloud purchased are more than B cloud, A cloud server receives 70% of user requests, processes read and write operations of users, B cloud server receives 30% of user requests, and processes read operations of users. Since it is a master-slave architecture, A cloud server with more resources can be the main service provider, referred to as "master server", and B cloud server with less resources is referred to as "slave server". In the master-slave architecture, the master and slave servers can receive user requests, but the data write (add, delete, modify) can only operate the database of A cloud server, and the data read operation can be processed by the databases of A cloud server and B cloud server.

[0047] However, in the related art, since the system design is that A cloud server is responsible for 70% of requests and B cloud server is responsible for 30% of requests, if the actual load of the database of A cloud server is 85% and the load of the database of B cloud server is 15% at this time, 15% of read requests in 30% of requests of B cloud server load directly operate the database of B cloud server, and 15% of database write requests are sent to the database of A cloud server. When the database write request in user request increases (such as read: write ratio = 1:1), it can cause the server resource utilization rate of A cloud to be close to or even reach 100%, and the server resource utilization rate of B cloud is only 15%, so that the database load of A cloud server responsible for writing is too high, which blocks read and write and affects business requests, and the database of B cloud server responsible for reading can be idle due to fewer read requests.

[0048] Therefore, the present disclosure proposes to realize the dynamic load balancing of database reading by deploying a software package file (such as a JAR package) on both the master server and the slave server; wherein the JAR package is a Java Archive file for connecting and operating a database, which contains the driver and related class library between the Java program and the database, so that the developer can use the functions of the database in the Java application, such as connecting the database, executing SQL queries, inserting, updating and deleting data, etc.; and the JAR file (Java Archive) is a software package file format, which is usually used to aggregate a large number of Java class files, related metadata and resource files into a file, in order to develop Java platform application software or library.

[0049] The main functions of the JAR package include: providing database connection function: the included classes and interfaces enable Java programs to connect to databases, which is the basis for implementing database operations; providing database operation function: defining how to operate the database, such as executing SQL statements and processing query results; supporting database transaction processing: providing support for database transactions to ensure database consistency.

[0050] Specifically, the following functions can be achieved in the present disclosure by using the software package file:

[0051] 1. Timely monitor the read / write frequency of the database and the CPU (Central Processing Unit), IO (Input / Output), and memory usage of the server.

[0052] 2. Complete the database read / write separation through the ShardingSphere-JDBC (database middleware) introduced thereby, wherein the ShardingSphere-JDBC is a lightweight Java framework that can provide additional services at the JDBC (Java Database Connectivity) layer.

[0053] 3. Dynamically manage the ShardingSphere-JDBC database configuration through the Zookeeper (an open source distributed coordination service software) introduced thereby; wherein the main functions provided by the Zookeeper can include data publishing / subscription, load balancing, naming service, distributed coordination / notification, cluster management, Master election, distributed lock and distributed queue, etc.

[0054] 4. Dynamically allocate the proportion of data read operations on the master server and the slave server through the monitoring of the server resource usage.

[0055] 5. Provide a configuration portal for the operation and maintenance personnel to modify the pre-processing and distribution of weight instructions of the master-slave server database.

[0056] As shown in FIG. 2, the present disclosure deploys a software package file in both the master and slave servers, for example, the A cloud server (master node / first database node) processes 70% of the requests, and the B cloud server (slave node / second database node) processes 30% of the requests; among the 30% of the requests of the B cloud server, 15% of the read requests are directly operated on the database of the B cloud server, and 15% of the database write operations are sent to the database of the A cloud server; the A cloud server processes 70% of the requests, but since the database of the B cloud server can only process 30% of the requests, it has already processed 15% of the read requests from the B cloud server, at this time, the A cloud server can send 15% of its read requests to the database of the B cloud server according to the functions provided by the software package file.

[0057] The present disclosure is further described below in conjunction with the accompanying drawings:

[0058] Referring to FIG. 3, a flowchart of the steps of the load balancing method based on a database cluster provided by an embodiment of the present disclosure is shown, the database cluster includes a first database node and a second database node, the first database node is used to process write requests and read requests, and the second database node is used to process read requests.

[0059] In some embodiments of the present disclosure, the first database node and the second database node are database nodes in different cloud platforms.

[0060] As some examples, since the server fees provided by each cloud platform operator are different, the database service can be deployed in the server resources provided by one or more cloud platform operators; taking the purchase of server resources of the A cloud platform operator and the server resources of the B cloud platform operator as an example, the number of purchased server resources can be used as the primary service provider, i.e., the first database node, and the number of purchased server resources can be used as the secondary service provider, i.e., the second database node, for example, if the number of purchased server resources of the A cloud platform operator is the most, then the A cloud is the first database node, and if the number of purchased server resources of the B cloud platform operator is the least, then the B cloud is the second database node.

[0061] Among the first database node or the second database node, there is one or more databases, by determining the number of databases in the first database node and the second database node, a database cluster is established; for example, there are five databases in the first database node to provide services, and there are three databases in the second database node to provide services, so there are eight databases in the database cluster.

[0062] As some examples, since the first database node provides the most server resources, it can be used to be responsible for processing write requests and read requests in the business, and the second database node provides the least server resources, and is used to be responsible for processing read requests in the business.

[0063] In actual application, after the cloud platform server deploys the service, the corresponding business can be provided for the user, and when the user handles the business, the corresponding access request such as insert, delete, update, and select request can be generated, the select request without modification can be regarded as a read request, and the insert, delete, and update can be regarded as a write request.

[0064] Specifically, the method can include the following steps:

[0065] In step 301, an access request for the database cluster is received, the access request is distributed to the first database node and the second database node, and all write requests in the first access request distributed to the second database node are migrated to the first database node.

[0066] As some examples, taking providing a cloud storage service to a user as an example, the service is deployed in a database cluster composed of a first database node and a second database node, the user can initiate an access request for the database cluster according to the demand, and when the access request for the database cluster is received, it is distributed to the first database node and the second database node for processing according to the domain name server.

[0067] In some examples, the first access request is an access request sent to the second database node, since the second database node only processes read requests in the business, the first database node is responsible for all write requests, and can also process read requests, after the second database node receives the first access request, all write requests in the first access request can be transferred to the first database node for processing through the read-write separation function provided by the database middleware, and the read request in the first access request is processed through the second database node.

[0068] In some embodiments of the present disclosure, the distribution of the access request to the first database node and the second database node includes: distributing the access request to the first database node and the second database node according to the current configured access request distribution ratio.

[0069] As some examples, the access request distribution ratio is determined according to the sum of the number of all servers in the first database node and the second database node under the database cluster, and the operation and maintenance personnel can manually configure, for example, there are 7 servers in the first database node and 3 servers in the second database node, and the access request distribution ratio is 7:3.

[0070] As some examples, after receiving the access request for the database cluster, the access request can be allocated to the first database node and the second database node according to the number of access requests and the currently configured access request allocation ratio. Specifically, for example, if the number of access requests is 100 and the access request allocation ratio is 7:3, 70 access requests can be allocated to the first database node for processing, and 30 access requests can be allocated to the second database node for processing.

[0071] In some embodiments of the present disclosure, the method further includes modifying the currently configured access request allocation ratio in response to a ratio modification request.

[0072] As some examples, if the number of servers that can provide services in the first database node or the second database node changes, the currently configured access request allocation ratio needs to be modified. For example, if the access request allocation ratio is 7:3, the number of servers that can provide services in the first database node and the second database node changes, there are 5 servers in the first database node and 5 servers in the second database node, and the operation and maintenance personnel can manually configure the access request allocation ratio to be modified to 5:5.

[0073] In actual applications, after the currently configured access request allocation ratio is modified, subsequent access requests will be allocated to the first database node and the second database node according to the modified access request allocation ratio.

[0074] In some embodiments of the present disclosure, after the access request is allocated to the first database node and the second database node, the method further includes separating read requests and write requests in the allocated access requests in the first database node and the second database node.

[0075] As some examples, a software package file can be deployed in the first database node and the second database node, and ShardingSphere-JDBC (a database middleware) introduced by the software package file can be used to complete database read-write separation. Specifically, when the database middleware uses the read-write separation function, a cluster mode can be started, Zookeeper (a coordination service software) can be used as a configuration management center, the second database node is responsible for read requests, and the first database node also supports reading to be started at the same time, so that the database of the first database node is responsible for all write requests and can also process read requests.

[0076] In some examples, the access request allocated to the second database node can be taken as the first access request, and the access request allocated to the first database node can be taken as the second access request; the first database node and the second database node can separate read and write of read requests and write requests in the allocated access requests through database middleware; as shown in FIG. 2, the database middleware introduced by the software package file in the first database node (A cloud) can separate read and write of read requests and write requests in the second access request, and the database middleware introduced by the software package file in the second database node (B cloud) can separate read and write of read requests and write requests in the first access request.

[0077] In some embodiments of the present disclosure, the method further includes synchronizing data in the first database node to the second database node.

[0078] As some examples, since the write requests of the user are all processed through the first database node, and the read requests of the user can be processed in the first database node or in the second database node, in order to maintain the unity of data, the first database node can be connected with the second database node through a network dedicated line, and data in the first database node can be synchronized to the second database node in real time.

[0079] In step 302, the current server resource usage corresponding to the first database node is obtained, and at least part of read requests in the second access request allocated to the first database node are migrated to the second database node according to the current server resource usage.

[0080] In some embodiments of the present disclosure, the current server resource usage includes a current server resource usage rate.

[0081] As some examples, the current server resource usage of the first database node can be queried by using SQL (Structured Query Language) to determine the current server resource usage (such as central processing unit, input / output, memory usage), so as to determine whether to migrate the access request and whether to modify the read request allocation ratio.

[0082] In some examples, if the current server resource usage rate is greater than a preset server resource usage rate, at least part of read requests in the second access request allocated to the first database node can be migrated to the second database node, so as to reduce the server resource usage rate corresponding to the first database node.

[0083] In some embodiments of the present disclosure, the migrating, according to the current server resource usage, at least part of read requests in the second access requests allocated to the first database node to the second database node comprises: modifying, according to the current server resource usage, a read request allocation ratio of the first database node and the second database node, and migrating, according to the modified read request allocation ratio, read requests in the second access requests allocated to the first database node to the second database node.

[0084] In some embodiments of the present disclosure, before the modifying, according to the current server resource usage, the read request allocation ratio of the first database node and the second database node, the method further comprises: judging whether the current server resource usage rate is greater than a preset server resource usage rate; and when the current server resource usage rate is greater than the preset server resource usage rate, performing the modifying, according to the current server resource usage, the read request allocation ratio of the first database node and the second database node.

[0085] As some examples, an operation and maintenance personnel can introduce a Zookeeper (coordination service software) through a software package file to pre-set a read request allocation ratio of the first database node and the second database node, and migrate, through the read request allocation ratio, read requests in the second access requests allocated to the first database node to the second database node. For example, the first database node is DB1, the second database node is DB2, and the read request allocation ratio can be set as DB1:DB2=3:7.

[0086] In some examples, after obtaining the current server resource usage corresponding to the first database node, when it is determined that the current server resource usage rate is greater than the preset server resource usage rate, the operation and maintenance personnel can modify, according to the current server resource usage, the pre-set read request allocation ratio of the first database node and the second database node, and migrate, according to the modified read request allocation ratio, read requests in the second access requests allocated to the first database node to the second database node. For example, the current server resource usage rate is 85%, the preset server resource usage rate is 70%, and in order to balance the load of the first database node, the read requests of the first database node need to be reduced. Therefore, the operation and maintenance personnel can modify the read request allocation ratio as DB1:DB2=2:8.

[0087] As some examples, the Zookeeper (coordination service software) can provide a TCP (Transmission Control Protocol) connection mode for the operation and maintenance personnel to modify the read request allocation ratio.

[0088] As some examples, the preset server resource usage rate can be 70% or 80%, etc., which can be determined by the skilled person in the art by performing performance stress testing on the first database node and the second database node, and the present disclosure does not limit this.

[0089] In some embodiments of the present disclosure, the current server resource usage includes a current server resource usage rate, and the modifying of the read request allocation ratio of the first database node and the second database node according to the current server resource usage includes: obtaining a current access request frequency of the first database node; and modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage rate and the current access request frequency.

[0090] As some examples, the current access request frequency of the first database node or the second database node can be queried by using an SQL (Structured Query Language) query; by using the structured query language, the number of executions of various types of commands since the first database node or the second database node is started can be queried, including insert, delete, update, and select, and the insert, delete, and update are regarded as writing, and the select is regarded as reading; specifically, the number of times of writing requests and the number of times of reading requests in 1 minute can be known by querying 2 times at an interval of 1 minute, and the current access request frequency of the first database node or the second database node can be obtained by performing the query operation every 1 minute.

[0091] In some examples, after the current access request frequency of the first database node is determined, if it is observed that the writing requests in the current access request frequency gradually increase, or the current server resource usage rate increases, and one of them reaches a threshold value (a threshold value of the preset access request frequency or the preset server resource usage rate), the reading requests of the first database node are reduced, and the limited resources of the first database node are preferentially used to satisfy the writing requests; for example, if the current server resource usage rate exceeds the preset server resource usage rate (85%) due to the increase of the writing requests of the first database node, the reading requests of the first database node are reduced; if the current server resource usage rate exceeds 95%, the reading requests are all migrated to the second database node, etc.

[0092] As some examples, to implement migration of read requests in a second access request assigned to the first database node, the preset read request assignment ratio in Zookeeper (coordination service software) can be changed. For example, the first database node is DB1, the second database node is DB2, and the preset read request assignment ratio is set to DB1:DB2=3:7. When the read requests of the first database node are to be reduced, an operator can modify the configuration to DB1:DB2=2:8.

[0093] In actual applications, an operator can dynamically modify the read request assignment ratio of the first database node and the second database node according to the current server resource usage and the current access request frequency, such as increasing or reducing the read request assignment ratio of the first database node and the second database node.

[0094] In some embodiments of the present disclosure, modifying the read request assignment ratio of the first database node and the second database node according to the current server resource usage and the current access request frequency includes: obtaining a test result of the first database node; wherein the test result includes a correspondence between server resource usage and database read-write frequency; determining to-be-migrated read request information according to the current server resource usage, the current access request frequency, and the correspondence between server resource usage and database read-write frequency; and modifying the read request assignment ratio of the first database node and the second database node according to the to-be-migrated read request information.

[0095] As some examples, after purchasing server resources, performance stress testing can be performed on the first database node and the second database node, that is, a tool is used to perform uninterrupted and different-frequency read-write operations on the databases under the first database node and the second database node, while monitoring various indicators (such as CPU, IO, memory, etc.) of the servers on which the first database node and the second database node are located, testing the database read-write frequency that can be supported when the resource usage rate is from 50% to 100% (with an interval of 5% increase), and determining the test result of the first database node and the second database node, and recording it into a test table for subsequent use.

[0096] The test table is used to determine the consumption of server resources (CPU, IO, memory, etc.) of different read-write conditions of the database; test conditions are, for example, read only, read (80%) + write (20%), read (50%) + write (50%), and write only; as shown in FIG. 2, the database of the A cloud (the first database node) can support 70% read-write, and the database of the B cloud (the second database node) can support 30% read; these numbers are determined by the test table; at the same time, the read and write of the database consume different server resources, and generally, write is more than read; when the write request in the subsequent access request increases, in order to ensure the stability of the first database node, when the second database node has only 5% of the write request migrated to the first database node, the first database node can send more than 5% of the read request to the second database node, thereby reducing the consumption of resources of the first database node; however, the specific value of "more than 5%" needs to be determined by the test table.

[0097] In some examples, the read request information to be migrated can be determined according to the proportion of write requests migrated from the second database node to the first database node; as shown in FIG. 2, the proportion of write requests migrated from the second database node to the first database node is 15%, and the read request information to be migrated can be greater than or equal to 15%.

[0098] In some examples, the read request information to be migrated can also be the number of read requests; specifically, when the server resource usage is 85%, the server can withstand an access request frequency of 1000, the current server resource usage corresponding to the first database node is 85%, and the current access request frequency is 1200; the 200 excess access requests are the read request information to be migrated, and 200 read requests can be migrated to the first database node; according to the read request information to be migrated, the operation and maintenance personnel can modify the preset read request distribution ratio of the first database node and the second database node through Zookeeper (coordination service software), for example, reduce the read ratio of the first database node and increase the read ratio of the second database node, so that the master node reduces the read request and the slave node increases the read request, to realize load balancing of the first database node.

[0099] For example, the first database node is DB1, the second database node is DB2, and the preset read request distribution ratio is set to DB1:DB2=3:7; when the read request of the first database node is to be reduced, the operation and maintenance personnel can modify the configuration to DB1:DB2=2:8.

[0100] In the embodiments of the present disclosure, by receiving an access request for a database cluster, assigning the access request to a first database node and a second database node, and migrating all write requests in the first access request assigned to the second database node to the first database node, the current server resource usage corresponding to the first database node is obtained, and at least part of the read requests in the second access request assigned to the first database node are migrated to the second database node according to the current server resource usage, so that the read requests of the first database node are migrated to the second database node according to the current server resource usage, the load of the first database node is reduced, the resource usage of the second database node is improved, and the load balancing of the read requests in the database cluster is realized.

[0101] Referring to FIG. 4, a flowchart of steps of another database cluster-based load balancing method provided by an embodiment of the present disclosure is shown, the database cluster including a first database node and a second database node, the first database node being used for processing write requests and read requests, and the second database node being used for processing read requests.

[0102] In some embodiments of the present disclosure, the first database node and the second database node are database nodes in different cloud platforms.

[0103] As some examples, since the server fees provided by each cloud platform operator are different, the database service can be deployed in the server resources provided by one or more cloud platform operators; taking the purchased server resources of A cloud platform operator and B cloud platform operator as examples, the one with more purchased server resources can be taken as the primary service provider, i.e., the first database node, and the one with less purchased server resources can be taken as the secondary service provider, i.e., the second database node, for example, if the purchased server resources of A cloud platform operator are the most, then A cloud is the first database node, and if the purchased server resources of B cloud platform operator are the least, then B cloud is the second database node.

[0104] Among the first database node or the second database node, there is one or more databases, by determining the number of databases in the first database node and the second database node, a database cluster is established; for example, there are five databases in the first database node providing services, and there are three databases in the second database node providing services, so there are eight databases in the database cluster.

[0105] As some examples, since the first database node provides the most server resources, it can be used to be responsible for processing write requests and read requests in the business, and the second database node provides the least server resources, and is used to be responsible for processing read requests in the business.

[0106] In actual application, after the cloud platform server deploys the service, the corresponding business can be provided for the user, and when the user handles the business, the corresponding access request such as insert, delete, update and select request can be generated, the select request without modification can be regarded as a read request, and the insert, delete and update request can be regarded as a write request.

[0107] In step 401, an access request for the database cluster is received, the access request is allocated to the first database node and the second database node, and all write requests in the first access request allocated to the second database node are migrated to the first database node.

[0108] As some examples, taking the cloud storage service provided for the user as an example, the service is deployed in the database cluster composed of the first database node and the second database node, the user can initiate the access request for the database cluster according to the demand, and when the access request for the database cluster is received, the access request is allocated to the first database node and the second database node for processing according to the domain name server.

[0109] In some examples, the first access request is the access request sent to the second database node, since the second database node only processes the read request in the business, the first database node is responsible for all write requests and can also process the read request, after the second database node receives the first access request, all write requests in the first access request can be transferred to the first database node for processing through the read-write separation function provided by the database middleware, and the read request in the first access request is processed through the second database node.

[0110] In step 402, the current server resource usage corresponding to the first database node is obtained.

[0111] In some embodiments of the present disclosure, the current server resource usage includes the current server resource usage rate.

[0112] As some examples, the SQL (Structured Query Language) can be used to query the current server resource usage rate (such as the central processing unit, input / output, memory usage) corresponding to the first database node, so as to determine the current server resource usage; wherein the current server resource usage is used to determine whether to migrate the access request and whether to modify the read request allocation ratio.

[0113] In step 403, the current access request frequency of the first database node is obtained.

[0114] As some examples, the current access request frequency of the first database node or the second database node can be queried using a structured query language; by using the structured query language, the execution times of various types of commands since the first database node or the second database node is started can be queried, including insert, delete, update, and select, taking insert, delete, and update as write and taking select as read; specifically, the query operation can be performed twice every 1 minute, and the total number of write requests and the total number of read requests in the 1 minute can be known by the difference; by performing the query operation every 1 minute, the current access request frequency of the first database node or the second database node can be obtained.

[0115] In step 404, a test result of the first database node is obtained; the test result includes a correspondence between a server resource usage and a database read-write frequency.

[0116] As some examples, after the server resources are purchased, performance stress testing operations can be performed on the first database node and the second database node, that is, the databases under the first database node and the second database node are continuously read and written at different frequencies by using a tool, and various indicators (such as CPU, IO, memory, etc.) of the server where the first database node and the second database node are located are monitored, the resource usage rate is tested from 50% to 100% (increased by 5% each time), and the database read-write frequency that can be supported is determined, so as to determine the test result of the first database node and the second database node, and record the test result into a test table for subsequent use.

[0117] The test table is used to determine the consumption (CPU, IO, memory, etc.) of the server resources by the database under different read-write conditions; the test conditions are, for example, only read, read (80%) + write (20%), read (50%) + write (50%), and only write; as shown in FIG. 2, the database of the A cloud (the first database node) can support 70% read-write, and the database of the B cloud (the second database node) can support 30% read; these numbers are determined by the test table; in addition, the read and write of the database consume different resources of the server, and generally, more write and less read; when the write request in the subsequent access request increases, in order to ensure the stability of the first database node, when the second database node has only 5% write request migrated to the first database node, the first database node can send more than 5% read request to the second database node, so as to reduce the consumption of the resources of the first database node; however, the specific value of "more than 5%" needs to be determined by the test table.

[0118] At step 405, the read request information to be migrated is determined according to the current server resource usage, the current access request frequency, and the corresponding relationship between server resource usage and database read-write frequency.

[0119] In some examples, the read request information to be migrated can be determined according to the proportion of write requests migrated from the second database node to the first database node; for example, as shown in FIG. 2, the proportion of write requests migrated from the second database node to the first database node is 15%, and the read request information to be migrated can be greater than or equal to 15%.

[0120] At step 406, the read request allocation ratio of the first database node and the second database node is modified according to the read request information to be migrated.

[0121] In some examples, the read request information to be migrated can also be the number of read requests; specifically, when the server resource usage is 85%, the server can withstand an access request frequency of 1000, the current server resource usage of the first database node is 85%, and the current access request frequency is 1200, the 200 excess access requests are the read request information to be migrated, and 200 read requests can be migrated to the first database node. According to the read request information to be migrated, the operation and maintenance personnel can modify the preset read request allocation ratio of the first database node and the second database node through Zookeeper (coordination service software), such as reducing the read ratio of the first database node and increasing the read ratio of the second database node, so that the master node reduces read requests and the slave node increases read requests, to achieve load balancing of the first database node.

[0122] For example, when the first database node is DB1 and the second database node is DB2, and the preset read request allocation ratio is set to DB1:DB2=3:7, the operation and maintenance personnel can modify the configuration to DB1:DB2=2:8 to reduce the read requests of the first database node.

[0123] At step 407, the read requests of the second access requests allocated to the first database node are migrated to the second database node according to the modified read request allocation ratio.

[0124] As some examples, the operation and maintenance personnel can introduce Zookeeper (coordination service software) through a software package file to pre-set the read request allocation ratio of the first database node and the second database node, and migrate the read requests of the second access requests allocated to the first database node to the second database node through the read request allocation ratio; for example, when the first database node is DB1 and the second database node is DB2, the read request allocation ratio can be set to DB1:DB2=3:7.

[0125] In some examples, after obtaining the current server resource usage corresponding to the first database node, when it is determined that the current server resource usage rate is greater than the preset server resource usage rate, the operation and maintenance personnel can modify the read request allocation ratio previously set for the first database node and the second database node according to the current server resource usage, and migrate the read requests in the second access requests allocated to the first database node to the second database node according to the modified read request allocation ratio; for example, when the current server resource usage rate is 85% and the preset server resource usage rate is 70%, in order to balance the load of the first database node, the read requests of the first database node need to be reduced, and the operation and maintenance personnel can modify the read request allocation ratio to DB1:DB2=2:8.

[0126] As some examples, the Zookeeper (coordination service software) can provide a way for the operation and maintenance personnel to modify the read request allocation ratio through a TCP (Transmission Control Protocol) connection.

[0127] As some examples, the preset server resource usage rate can be 70% or 80%, etc., which can be determined by the performance stress test operation on the first database node and the second database node by those skilled in the art, and the present disclosure does not limit this.

[0128] In the embodiments of the present disclosure, by receiving an access request for the database cluster, allocating the access request to the first database node and the second database node, and migrating all write requests in the first access requests allocated to the second database node to the first database node, obtaining the current server resource usage corresponding to the first database node, obtaining the current access request frequency of the first database node, obtaining the test result of the first database node, wherein the test result includes the correspondence between the server resource usage and the database read-write frequency, determining the read request information to be migrated according to the current server resource usage rate, the current access request frequency, and the correspondence between the server resource usage and the database read-write frequency, modifying the read request allocation ratio of the first database node and the second database node according to the read request information to be migrated, and migrating the read requests in the second access requests allocated to the first database node to the second database node according to the modified read request allocation ratio, the read requests of the first database node are migrated to the second database node according to the current server resource usage, the load of the first database node is reduced, the resource usage rate of the second database node is improved, and the load balancing of the read requests in the database cluster is realized.

[0129] It should be noted that, for the method embodiments, the series of acts combined is described for simplicity, but those skilled in the art should know that the disclosure embodiments are not limited to the order of acts described, because according to the disclosure embodiments, certain steps can be performed in other orders or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the acts involved are not necessarily the disclosure embodiments.

[0130] Referring to FIG. 5, a structural schematic diagram of a load balancing device based on a database cluster is shown, the database cluster includes a first database node and a second database node, the first database node is used for processing write requests and read requests, and the second database node is used for processing read requests, and can specifically include the following modules:

[0131] The access request distribution module 501 is configured to receive an access request for the database cluster, distribute the access request to the first database node and the second database node, and migrate all write requests in the first access request distributed to the second database node to the first database node.

[0132] The read request migration module 502 is configured to obtain a current server resource usage corresponding to the first database node, and migrate at least part of read requests in the second access request distributed to the first database node to the second database node according to the current server resource usage.

[0133] In some embodiments of the disclosure, the at least part of read requests in the second access request distributed to the first database node are migrated to the second database node according to the current server resource usage, including:

[0134] According to the current server resource usage, the read request distribution ratio of the first database node and the second database node is modified, and the read requests in the second access request distributed to the first database node are migrated to the second database node according to the modified read request distribution ratio.

[0135] In some embodiments of the disclosure, the current server resource usage includes a current server resource usage rate, and the read request distribution ratio of the first database node and the second database node is modified according to the current server resource usage, including:

[0136] The current access request frequency of the first database node is obtained.

[0137] modify a read request distribution ratio of the first database node and the second database node according to the current server resource usage and the current access request frequency.

[0138] In some embodiments of the present disclosure, the modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage and the current access request frequency comprises:

[0139] obtaining a test result of the first database node; wherein the test result comprises a correspondence between server resource usage and database read-write frequency;

[0140] determining read request information to be migrated according to the current server resource usage, the current access request frequency, and the correspondence between server resource usage and database read-write frequency;

[0141] modifying the read request distribution ratio of the first database node and the second database node according to the read request information to be migrated.

[0142] In some embodiments of the present disclosure, the current server resource usage comprises a current server resource usage rate.

[0143] Before the modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage, the method further comprises:

[0144] determining whether the current server resource usage rate is greater than a preset server resource usage rate;

[0145] when the current server resource usage rate is greater than the preset server resource usage rate, performing the modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage.

[0146] In some embodiments of the present disclosure, the distributing the access request to the first database node and the second database node comprises:

[0147] distributing the access request to the first database node and the second database node according to a currently configured access request distribution ratio.

[0148] In some embodiments of the present disclosure, the apparatus further comprises:

[0149] an access request distribution ratio modification module, configured to modify a currently configured access request distribution ratio in response to a ratio modification request.

[0150] In some embodiments of the present disclosure, the apparatus further comprises:

[0151] The read-write separation module is configured to separate read requests and write requests in the assigned access requests in the first database node and the second database node.

[0152] In some embodiments of the present disclosure, the apparatus further comprises:

[0153] The data synchronization module is configured to synchronize data in the first database node to the second database node.

[0154] In some embodiments of the present disclosure, the first database node and the second database node are database nodes in different cloud platforms.

[0155] In the embodiments of the present disclosure, by receiving an access request for a database cluster, assigning the access request to a first database node and a second database node, and migrating all write requests in the first access request assigned to the second database node to the first database node, the current server resource usage corresponding to the first database node is obtained, and at least part of the read requests in the second access request assigned to the first database node are migrated to the second database node according to the current server resource usage, so that the read requests of the first database node are migrated to the second database node according to the current server resource usage, the load of the first database node is reduced, the resource usage rate of the second database node is improved, and the load balancing of the read requests in the database cluster is realized.

[0156] An embodiment of the present disclosure further provides an electronic device, which can include a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and the computer program is executed by the processor to implement the above load balancing method based on a database cluster.

[0157] An embodiment of the present disclosure further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above load balancing method based on a database cluster.

[0158] For the apparatus embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts are referred to the part of the method embodiment.

[0159] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment are referred to each other.

[0160] Those skilled in the art will appreciate that embodiments of the disclosure can be supplied as a method, a device, or a computer program product. Thus, embodiments of the disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the disclosure can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0161] Embodiments of the disclosure are described in reference to flowchart illustrations and / or block diagrams of the methods, terminal devices (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, an embedded processor or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0162] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions means which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices to cause a series of operational steps to be performed on the computer or other programmable terminal devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0164] Although preferred embodiments of the disclosure have been described, those skilled in the art will appreciate that additional modifications and changes can be made to these embodiments without departing from the scope of the disclosure. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the disclosure.

[0165] Finally, it needs to be pointed out that in this article, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or terminal device including the above element.

[0166] The above provides a detailed introduction to the database cluster-based load balancing method, device, equipment and medium, this article applies specific examples to explain the principles and implementation modes of the present disclosure, the above example is only used to help understand the method and core idea of the present disclosure; at the same time, for those skilled in the art, according to the idea of the present disclosure, the specific implementation mode and application range will be changed, and the above description should not be understood as a limitation of the present disclosure.

Claims

1. A method for load balancing based on a database cluster, the database cluster comprising a first database node and a second database node, the first database node being configured to process write requests and read requests, and the second database node being configured to process read requests, the method comprising: receiving access requests for the database cluster, distributing the access requests to the first database node and the second database node, and migrating all write requests in first access requests distributed to the second database node to the first database node; obtaining a current server resource usage corresponding to the first database node, and migrating at least part of read requests in second access requests distributed to the first database node to the second database node according to the current server resource usage. The migrating at least part of read requests in second access requests distributed to the first database node to the second database node according to the current server resource usage comprises: modifying a read request distribution ratio of the first database node and the second database node according to the current server resource usage, and migrating read requests in second access requests distributed to the first database node to the second database node according to the modified read request distribution ratio. The current server resource usage comprises a current server resource usage rate.

2. The method of claim 1, wherein, The modifying a read request distribution ratio of the first database node and the second database node according to the current server resource usage comprises: obtaining a current access request frequency of the first database node; and modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage rate and the current access request frequency. The modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage rate and the current access request frequency comprises: obtaining a test result of the first database node, wherein the test result comprises a correspondence between a server resource usage and a database read-write frequency; determining read request information to be migrated according to the current server resource usage rate, the current access request frequency, and the correspondence between the server resource usage and the database read-write frequency; and modifying the read request distribution ratio of the first database node and the second database node according to the read request information to be migrated.

3. The method of claim 2, wherein, The current server resource usage comprises a current server resource usage rate. Before the modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage, the method further comprises: determining whether the current server resource usage rate is greater than a preset server resource usage rate; and in response to the current server resource usage rate being greater than the preset server resource usage rate, performing the modifying the read request distribution ratio of the first database node and the second database node according to the current server resource usage. ​ 4. The method of claim 3, wherein, ​ ​ ​ ​ 5. The method according to any one of claims 1 to 4, wherein, ​ ​ ​ ​ 6. The method according to any one of claims 1 to 4, wherein, The assigning the access request to the first database node and the second database node comprises: assigning the access request to the first database node and the second database node according to a current configured access request distribution ratio.

7. The method of claim 6, further comprising: modifying the current configured access request distribution ratio in response to a ratio modification request.

8. The method according to any one of claims 1 to 4, wherein, After the assigning the access request to the first database node and the second database node, the method further comprises: separating read requests and write requests in the assigned access requests in the first database node and the second database node.

9. The method of any one of claims 1 to 4, further comprising: synchronizing data in the first database node to the second database node.

10. The method according to any one of claims 1 to 4, wherein, The first database node and the second database node are database nodes in different cloud platforms.

11. A load balancing apparatus based on a database cluster, the database cluster comprising a first database node and a second database node, the first database node being configured to handle write requests and read requests, the second database node being configured to handle read requests, the apparatus comprising: an access request assigning module configured to receive an access request for the database cluster, assign the access request to the first database node and the second database node, and migrate all write requests in the first access request assigned to the second database node to the first database node; a read request migrating module configured to obtain a current server resource usage corresponding to the first database node, and migrate at least part of read requests in the second access request assigned to the first database node to the second database node according to the current server resource usage.

12. An electronic device, comprising a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program being executed by the processor to implement the method of any one of claims 1 to 10.

13. A computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method of any one of claims 1 to 10.

Images