Load balancing method and device based on database cluster, equipment and medium

Abstract

The embodiment of the invention provides a load balancing method and device based on a database cluster, equipment and a medium, and the method comprises the steps: receiving an access request for the database cluster, and distributing the access request to a first database node and a second database node, migrating all write requests in the first access requests allocated to the second database node to the first database node; and obtaining a current server resource use condition corresponding to the first database node, and migrating at least part of the read requests in the second access requests allocated to the first database node to the second database node according to the current server resource use condition. Through the embodiment of the invention, the read request of the first database node is migrated to the second database node according to the current server resource use condition, so that the load of the first database node is reduced, the resource use rate of the second database node is improved, and the load balance of the read request in the database cluster is further realized.

Description

Technical Field

[0001] The present invention relates to the field of database load balancing, and in particular to a load balancing method, device, equipment and medium based on a database cluster. Background Art

[0002] At present, in order to improve the reliability, scalability and performance of database clusters in cloud environments, a master-slave dual-cloud architecture is generally built. In this architecture, the master database is responsible for processing read and write operations, while the slave database is responsible for processing read operations. In this way, the master database only needs to process write operations, and the written data will be automatically synchronized to the slave database.

[0003] However, when this method is used, when there are many write operations in the business, the database responsible for writing will be overloaded, blocking reading and writing and thus affecting business requests, while the reading database may have idle resources due to fewer read requests. Summary of the invention

[0004] In view of the above problems, a method, apparatus, device and medium for load balancing based on a database cluster are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] A load balancing method based on a database cluster, the database cluster comprising a first database node and a second database node, the first database node being used to process write requests and read requests, the second database node being used to process read requests, the method comprising:

[0006] receiving an access request for the database cluster, allocating the access request to the first database node and the second database node, and migrating all write requests in the first access request allocated to the second database node to the first database node;

[0007] A current server resource usage corresponding to the first database node is obtained, and according to the current server resource usage, at least part of the read requests in the second access request allocated to the first database node is migrated to the second database node.

[0008] Optionally, the migrating at least part of the read requests in the second access requests allocated to the first database node to the second database node according to the current server resource usage includes:

[0009] According to the current server resource usage, the read request allocation ratio of the first database node and the second database node is modified, and according to the modified read request allocation ratio, the read request in the second access request allocated to the first database node is migrated to the second database node.

[0010] Optionally, the current server resource usage includes a current server resource usage rate, and modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage includes:

[0011] Obtaining a current access request frequency of the first database node;

[0012] The read request allocation ratio of the first database node and the second database node is modified according to the current server resource usage rate and the current access request frequency.

[0013] Optionally, modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage rate and the current access request frequency includes:

[0014] Obtaining a test result for the first database node; wherein the test result includes a correspondence between server resource usage and database read and write frequency;

[0015] Determine the read request information to be migrated according to the current server resource usage rate, the current access request frequency, and the correspondence between the server resource usage and the database read and write frequency;

[0016] According to the read request information to be migrated, the read request allocation ratio of the first database node and the second database node is modified.

[0017] Optionally, the current server resource usage includes a current server resource usage rate;

[0018] Before modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage, the method further includes:

[0019] Determining whether the current server resource usage rate is greater than a preset server resource usage rate;

[0020] When the current server resource usage is greater than a preset server resource usage, modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage is performed.

[0021] Optionally, allocating the access request to the first database node and the second database node includes:

[0022] The access request is allocated to the first database node and the second database node according to the currently configured access request allocation ratio.

[0023] Optionally, it also includes:

[0024] In response to the ratio modification request, the currently configured access request allocation ratio is modified.

[0025] Optionally, after allocating the access request to the first database node and the second database node, the method further includes:

[0026] In the first database node and the second database node, read-write separation is performed on the read request and the write request in the allocated access request.

[0027] Optionally, it also includes:

[0028] Synchronize the data in the first database node to the second database node.

[0029] Optionally, the first database node and the second database node are database nodes in different cloud platforms.

[0030] A load balancing device based on a database cluster, the database cluster comprising a first database node and a second database node, the first database node being used to process write requests and read requests, the second database node being used to process read requests, the device comprising:

[0031] an access request allocation module, configured to receive access requests for the database cluster, allocate the access requests to the first database node and the second database node, and migrate all write requests in the first access request allocated to the second database node to the first database node;

[0032] A read request migration module is used to obtain the current server resource usage corresponding to the first database node, and according to the current server resource usage, migrate at least part of the second access requests allocated to the first database node to the second database node.

[0033] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method described above when executed by the processor.

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0035] The embodiments of the present invention have the following advantages:

[0036] In an embodiment of the present invention, by receiving an access request for a database cluster, the access request is allocated to a first database node and a second database node, and all write requests in the first access request allocated to the second database node are migrated to the first database node; the current server resource usage corresponding to the first database node is obtained, and according to the current server resource usage, at least part of the read requests in the second access request allocated to the first database node is migrated to the second database node, thereby achieving migration of the read request of the first database node to the second database node according to the current server resource usage, reducing the load of the first database node, and improving the resource utilization rate of the second database node, thereby achieving load balancing of read requests in the database cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0038] Figure 1 It is a schematic diagram of a system architecture in the related art;

[0039] Figure 2 is a schematic diagram of a system architecture provided by an embodiment of the present invention;

[0040] Figure 3 It is a flowchart of the steps of a load balancing method based on a database cluster provided by an embodiment of the present invention;

[0041] Figure 4 It is a flowchart of another load balancing method based on a database cluster provided by an embodiment of the present invention;

[0042] Figure 5 It is a structural block diagram of a load balancing device based on a database cluster provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In related technologies, such as Figure 1, the database service can be deployed on the cloud servers of two cloud service operators, namely A cloud server and B cloud server. Due to reasons such as server charges, the number of purchased two cloud servers is different. For example, if the server resources of A cloud are more than those of B cloud, A cloud server receives 70% of user requests and processes user read and write operations, and B cloud server receives 30% of user requests and processes user read operations. Due to the master-slave architecture, A cloud server with more resources can be used as the main service provider and is called the "master server", and B cloud server with fewer resources is called the "slave server"; in the master-slave architecture, both the master and slave servers can receive user requests, but data writing (addition, deletion, modification) can only operate the database of A cloud server, while data reading operations, the databases of A cloud server and B cloud server can process user read operations.

[0045] However, in the related art, since the system design is that cloud server A is responsible for 70% of the requests and cloud server B is responsible for 30% of the requests, if the actual load of the database of cloud server A is 85% at this time, and the load of the database of cloud server B is 15%, 15% of the read requests in the 30% requests of the load of cloud server B directly operate the database of cloud server B, and 15% of the database write requests are sent to the database of cloud server A. When the number of database write requests in user requests increases (such as when the read:write ratio = 1:1), the server resource utilization rate of cloud A may be close to or even reach 100%, while the server resource utilization rate of cloud B is only 15%, which makes the database of cloud server A responsible for writing too high, blocking reading and writing and thus affecting business requests, while the database of cloud server B responsible for reading may have idle resources due to fewer read requests.

[0046] Therefore, the present invention proposes that by deploying a software package file (such as a JAR package) on both the master and slave servers, dynamic load balancing of database reading can be achieved; wherein, the JAR package is a Java Archive file for connecting to and operating a database, which includes a driver program and related class libraries between a Java program and a database, so that developers can use database functions in Java applications, such as connecting to a database, executing SQL queries, inserting, updating and deleting data, etc.; and the JAR file (Java Archive) is a software package file format, which is usually used to aggregate a large number of Java class files, related metadata and resource files into one file to develop Java platform application software or libraries.

[0047] The main functions of the JAR package include: providing database connection function: the contained classes and interfaces enable Java programs to connect to the database, which is the basis for implementing database operations; providing database operation function: defining how to operate the database, such as executing SQL statements and processing query results; supporting database transaction processing: providing support for database transactions to ensure database consistency.

[0048] Specifically, the following functions can be accomplished in the present invention by utilizing the software package file:

[0049] 1. Regularly monitor the database read and write frequency and server CPU (Central Processing Unit), IO (Input / Output), and memory usage.

[0050] 2. Through the introduced ShardingSphere-JDBC (database middleware), the database read-write separation is completed. ShardingSphere-JDBC is a lightweight Java framework that can provide additional services at the JDBC (Java Database Connectivity) layer.

[0051] 3. Dynamically manage ShardingSphere-JDBC database configuration through the introduced Zookeeper (an open source distributed coordination service software); the main functions provided by Zookeeper include data publishing / subscription, load balancing, naming service, distributed coordination / notification, cluster management, Master (master node) election, distributed locks and distributed queues, etc.

[0052] 4. Dynamically allocate the ratio of data read operations on the master and slave servers by monitoring server resource usage.

[0053] 5. Provide configuration entry for operation and maintenance personnel to modify the preprocessing and distribution of weight instructions of the master-slave server database.

[0054] like Figure 2The present invention deploys a software package file on both the master and slave servers, such as A cloud server (master node / first database node) processes 70% of the requests, and B cloud server (slave node / second database node) processes 30% of the requests; 15% of the read requests of the 30% requests of B cloud server directly operate the database of B cloud itself, and 15% of the database write operations are sent to the database of A cloud; A cloud server processes 70% of the requests, but because the database of B cloud can only process 30% of the requests, it has currently processed 15% of the read requests from the B cloud server service. At this time, the service of A cloud server can separate 15% of its own read requests and send them to the database of B cloud server according to the functions provided by the software package file.

[0055] The present invention will be further described below in conjunction with the accompanying drawings:

[0056] Reference Figure 3 , showing a step flow chart of a load balancing method based on a database cluster provided in an embodiment of the present invention, wherein the database cluster includes a first database node and a second database node, wherein the first database node is used to process write requests and read requests, and the second database node is used to process read requests.

[0057] In some embodiments of the present invention, the first database node and the second database node are database nodes in different cloud platforms.

[0058] As some examples, since each cloud platform operator provides different server charges, the database service may be deployed in the server resources provided by one or more cloud platform operators; taking the server resources purchased from cloud platform operator A and cloud platform operator B as an example, the one with more purchased server resources can be used as the primary service provider, i.e., the first database node, and the one with fewer server resources can be used as the secondary service provider, i.e., the second database node. For example, if the server resources purchased from cloud platform operator A are the most, then A cloud is the first database node, and if the server resources purchased from cloud platform operator B are the least, then B cloud is the second database node.

[0059] Among them, there will be one or more databases in the first database node or the second database node. By determining the number of databases in the first database node and the second database node, a database cluster is established; if there are five databases providing services in the first database node and three databases providing services in the second database node, there are 8 databases in the database cluster.

[0060] As some examples, since the first database node provides the most server resources, it can be used to process write requests and read requests in the business, while the second database node provides the least server resources and is used to process read requests in the business.

[0061] In actual applications, after the service is deployed on the cloud platform server, corresponding services can be provided to users. When users handle services, corresponding access requests will be generated, such as insert, delete, update, and select requests. The behavior of querying without making modifications can be regarded as read requests, and the addition, deletion, and modification can be regarded as write requests.

[0062] Specifically, the following steps may be included:

[0063] Step 301: receive an access request for the database cluster, allocate the access request to the first database node and the second database node, and migrate all write requests in the first access request allocated to the second database node to the first database node.

[0064] As some examples, taking the provision of cloud storage services to users as an example, the service is deployed in a database cluster consisting of a first database node and a second database node. Users can initiate access requests to the database cluster according to their needs. When access requests to the database cluster are received, they are assigned to the first database node and the second database node for processing according to the domain name server.

[0065] In some examples, the first access request is an access request sent to the second database node. Since the second database node only processes read requests in the business, the first database node is responsible for all write requests and can also process read requests. After the second database node receives the first access request, it can transfer all write requests in the first access request to the first database node for processing through the read-write separation function provided by the database middleware, while the read requests in the first access request are processed by the second database node.

[0066] In some embodiments of the present invention, allocating the access request to the first database node and the second database node includes: allocating the access request to the first database node and the second database node according to a currently configured access request allocation ratio.

[0067] As some examples, the access request allocation ratio is determined based on the sum of the number of all servers in the first database node and the second database node under the database cluster. The operation and maintenance personnel can manually configure it. For example, if there are 7 servers in the first database node and 3 servers in the second database node, the access request allocation ratio is 7:3.

[0068] As some examples, after receiving access requests for a database cluster, the access requests can be allocated to the first database node and the second database node according to the number of access requests and the currently configured access request allocation ratio; specifically, if the number of access requests is 100 and the access request allocation ratio is 7:3, for example, 70 access requests can be allocated to the first database node for processing, and 30 access requests can be allocated to the second database node for processing.

[0069] In some embodiments of the present invention, the method further includes: modifying the currently configured access request allocation ratio in response to a ratio modification request.

[0070] As some examples, if the number of servers that can provide services in the first database node or the second database node changes, the currently configured access request allocation ratio needs to be modified; for example, if the access request allocation ratio is 7:3, and the number of servers that can provide services in the first database node and the second database node changes, there are 5 servers in the first database node and 5 servers in the second database node, the operation and maintenance personnel can manually configure and modify the access request allocation ratio to 5:5.

[0071] In actual applications, after the currently configured access request allocation ratio is modified, subsequent access requests will be allocated to the first database node and the second database node according to the modified access request allocation ratio.

[0072] In some embodiments of the present invention, after allocating the access request to the first database node and the second database node, the method further includes: performing read-write separation on the read request and the write request in the allocated access request in the first database node and the second database node.

[0073] As some examples, the software package file can be deployed on the first database node and the second database node, and the ShardingSphere-JDBC (database middleware) introduced by the software package file can be used to complete the database read-write separation; specifically, when the database middleware uses the read-write separation function, the cluster mode can be started, and Zookeeper (coordination service software) can be used as the configuration management center. The second database node is responsible for read requests, and the first database node is enabled to support reading, so that the database of the first database node is responsible for all write requests and can also process read requests.

[0074] In some examples, the access request assigned to the second database node can be used as the first access request, and the access request assigned to the first database node can be used as the second access request; the first database node and the second database node can perform read-write separation on the read request and the write request in the assigned access request through the database middleware; Figure 2 , the database middleware introduced by the software package file in the first database node (A cloud) can perform read-write separation on the read request and the write request in the second access request, and the database middleware introduced by the software package file in the second database node (B cloud) can perform read-write separation on the read request and the write request in the first access request.

[0075] In some embodiments of the present invention, the method further includes: synchronizing the data in the first database node to the second database node.

[0076] As some examples, since the user's write requests are all processed by the first database node, and the user's read requests can be processed by the first database node or the second database node, in order to maintain the consistency of the data, the first database node and the second database node can be connected through a dedicated network line, and the data in the first database node can be synchronized to the second database node in real time.

[0077] Step 302: Obtain current server resource usage corresponding to the first database node, and migrate at least part of the read requests in the second access requests allocated to the first database node to the second database node according to the current server resource usage.

[0078] In some embodiments of the present invention, the current server resource usage includes a current server resource usage rate.

[0079] As some examples, SQL (Structured Query Language) can be used to query the current server resource usage (such as central processing unit, input / output, and memory usage) corresponding to the first database node to determine the current server resource usage; wherein the current server resource usage is used to determine whether to migrate access requests and whether to modify the read request allocation ratio.

[0080] In some examples, if the current server resource usage is greater than the preset server resource usage, at least part of the read requests in the second access requests assigned to the first database node can be migrated to the second database node to reduce the server resource usage corresponding to the first database node.

[0081] In some embodiments of the present invention, migrating at least part of the read requests in the second access requests allocated to the first database node to the second database node according to the current server resource usage includes: modifying the read request allocation ratio between the first database node and the second database node according to the current server resource usage, and migrating the read requests in the second access requests allocated to the first database node to the second database node according to the modified read request allocation ratio.

[0082] In some embodiments of the present invention, before modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage, it also includes: determining whether the current server resource usage is greater than the preset server resource usage; and when the current server resource usage is greater than the preset server resource usage, executing the modification of the read request allocation ratio of the first database node and the second database node according to the current server resource usage.

[0083] As some examples, operation and maintenance personnel can pre-set the read request allocation ratio for the first database node and the second database node through Zookeeper (coordination service software) introduced by the software package file, and migrate the read request in the second access request allocated to the first database node to the second database node through the read request allocation ratio. For example, if the first database node is DB1 and the second database node is DB2, the read request allocation ratio can be set to DB1:DB2=3:7.

[0084] In some examples, after obtaining the current server resource usage corresponding to the first database node, when it is determined that the current server resource usage is greater than the preset server resource usage, the operation and maintenance personnel can modify the pre-set read request allocation ratio for the first database node and the second database node according to the current server resource usage, and migrate the read request in the second access request allocated to the first database node to the second database node according to the modified read request allocation ratio; if the current server resource usage is 85% and the preset server resource usage is 70%, in order to balance the load of the first database node, it is necessary to reduce the read requests of the first database node, and the operation and maintenance personnel can modify the read request allocation ratio to DB1:DB2=2:8.

[0085] As some examples, Zookeeper (coordination service software) can provide a TCP (Transmission Control Protocol) connection method for operation and maintenance personnel to modify the read request distribution ratio.

[0086] As some examples, the preset server resource utilization rate may be 70% or 80%, etc. Those skilled in the art may determine this by performing performance stress testing operations on the first database node and the second database node, and the present invention does not impose any limitation on this.

[0087] In some embodiments of the present invention, the current server resource usage includes a current server resource usage rate, and modifying the read request allocation ratio of the first database node and the second database node based on the current server resource usage includes: obtaining a current access request frequency of the first database node; and modifying the read request allocation ratio of the first database node and the second database node based on the current server resource usage rate and the current access request frequency.

[0088] As some examples, SQL (Structured Query Language) can be used to query the current access request frequency of the first database node or the second database node; by using the structured query language, the execution times of various types of commands since the first database node or the second database node was started can be queried, including insert, delete, update, and select, with insert, delete, and select being considered as writes and select as reads; specifically, the query can be performed twice at an interval of 1 minute, and the difference can be used to know the total number of write requests and read requests within the 1 minute, and the current access request frequency of the first database node or the second database node can be obtained by performing a query operation every 1 minute.

[0089] In some examples, after determining the current access request frequency of the first database node, if it is observed that the write requests in the current access request frequency gradually increase, or the current server resource utilization rate increases, and one of them reaches a threshold (a preset access request frequency or a preset server resource utilization rate threshold), the read requests of the first database node are reduced, and the limited resources of the first database node are used to satisfy the write requests first; if the increase in the write requests of the first database node causes the current server resource utilization rate to exceed the preset server resource utilization rate (85%), the read requests of the first database node are reduced; if the current server resource utilization rate exceeds 95%, all read requests are migrated to the second database node, etc.;

[0090] As some examples, if you want to migrate the read requests in the second access requests assigned to the first database node, you can do so by changing the preset read request allocation ratio in Zookeeper (coordination service software). For example, the first database node is DB1, the second database node is DB2, and the preset read request allocation ratio is set to DB1:DB2=3:7. When you want to reduce the read requests of the first database node, the operation and maintenance personnel can modify the configuration to DB1:DB2=2:8.

[0091] In actual applications, operation and maintenance personnel can dynamically modify the read request allocation ratio of the first database node and the second database node according to the current server resource utilization and the current access request frequency, such as increasing or decreasing the read request allocation ratio of the first database node and the second database node.

[0092] In some embodiments of the present invention, modifying the read request allocation ratio of the first database node and the second database node based on the current server resource utilization and the current access request frequency includes: obtaining a test result for the first database node; wherein the test result includes a correspondence between server resource utilization and database read and write frequency; determining read request information to be migrated based on the current server resource utilization, the current access request frequency, and the correspondence between server resource utilization and database read and write frequency; and modifying the read request allocation ratio of the first database node and the second database node based on the read request information to be migrated.

[0093] As some examples, after purchasing server resources, performance stress testing operations can be performed on the first database node and the second database node, that is, the databases under the first database node and the second database node are read and written uninterruptedly at different frequencies through tools, while monitoring various indicators of the servers where the first database node and the second database node services are located (such as: CPU, IO, memory, etc.), and testing resource utilization from 50% to 100% (increases at intervals of 5%) to see what database read and write frequencies can be supported respectively, thereby determining the test results of the first database node and the second database node, and recording them in a test table for subsequent use.

[0094] The test table is used to determine the consumption of server resources (CPU, IO, memory, etc.) by different read and write conditions of the database; test conditions include: read-only, read (80%) + write (20%), read (50%) + write (50%), write-only, etc. Figure 2In the figure, the database of cloud A (the first database node) can support a maximum of 70% reads and writes, and the database of cloud B (the second database node) can support a maximum of 30% reads. These figures are determined by test tables. At the same time, the read and write of the database consume different server resources. Generally, there are more writes than reads. When the write requests in subsequent access requests increase, in order to ensure the stability of the first database node, when only 5% of the write requests of the second database node migrate to the first database node, the first database node can send more than 5% of the read requests to the second database node, thereby reducing the consumption of resources of the first database node. The specific meaning of "more than 5%" here needs to be determined by the test table.

[0095] In some examples, the read request information to be migrated may be determined based on the ratio of write requests migrated from the second database node to the first database node; Figure 2 , the proportion of write requests migrated from the second database node to the first database node is 15%, and the proportion of read request information to be migrated may be greater than or equal to 15%.

[0096] In some examples, the read request information to be migrated may also be the number of read requests; specifically, when the server resource utilization rate is 85%, the access request frequency that the server can withstand is 1000, and the current server resource utilization rate corresponding to the first database node is 85%, and the current access request frequency is 1200, and the extra 200 access requests are the read request information to be migrated, and the 200 read requests can be migrated to the first database node. According to the read request information to be migrated, the operation and maintenance personnel can modify the preset read request allocation ratio of the first database node and the second database node through Zookeeper (coordination service software), such as reducing the read ratio of the first database node and increasing the read ratio of the second database node, so that the master node reduces read requests and the slave node increases read requests, so as to achieve load balancing for the first database node.

[0097] For example, if the first database node is DB1, the second database node is DB2, and the preset read request allocation ratio is set to DB1:DB2=3:7, when the read request of the first database node needs to be reduced, the operation and maintenance personnel can modify the configuration to DB1:DB2=2:8.

[0098] In an embodiment of the present invention, by receiving an access request for a database cluster, the access request is allocated to a first database node and a second database node, and all write requests in the first access request allocated to the second database node are migrated to the first database node; the current server resource usage corresponding to the first database node is obtained, and according to the current server resource usage, at least part of the read requests in the second access request allocated to the first database node is migrated to the second database node, thereby achieving migration of the read request of the first database node to the second database node according to the current server resource usage, reducing the load of the first database node, and improving the resource utilization rate of the second database node, thereby achieving load balancing of read requests in the database cluster.

[0099] Reference Figure 4 , showing a step flow chart of another load balancing method based on a database cluster provided by an embodiment of the present invention, wherein the database cluster includes a first database node and a second database node, the first database node is used to process write requests and read requests, and the second database node is used to process read requests.

[0100] In some embodiments of the present invention, the first database node and the second database node are database nodes in different cloud platforms.

[0101] As some examples, since each cloud platform operator provides different server charges, the database service may be deployed in the server resources provided by one or more cloud platform operators; taking the server resources purchased from cloud platform operator A and cloud platform operator B as an example, the one with more purchased server resources can be used as the primary service provider, i.e., the first database node, and the one with fewer server resources can be used as the secondary service provider, i.e., the second database node. For example, if the server resources purchased from cloud platform operator A are the most, then A cloud is the first database node, and if the server resources purchased from cloud platform operator B are the least, then B cloud is the second database node.

[0102] Among them, there will be one or more databases in the first database node or the second database node. By determining the number of databases in the first database node and the second database node, a database cluster is established; if there are five databases providing services in the first database node and three databases providing services in the second database node, there are 8 databases in the database cluster.

[0103] As some examples, since the first database node provides the most server resources, it can be used to process write requests and read requests in the business, while the second database node provides the least server resources and is used to process read requests in the business.

[0104] In actual applications, after the service is deployed on the cloud platform server, corresponding services can be provided to users. When users handle services, corresponding access requests will be generated, such as insert, delete, update, and select requests. The behavior of querying without making modifications can be regarded as read requests, and the addition, deletion, and modification can be regarded as write requests.

[0105] Step 401: receive an access request for the database cluster, distribute the access request to the first database node and the second database node, and migrate all write requests in the first access request distributed to the second database node to the first database node.

[0106] As some examples, taking the provision of cloud storage services to users as an example, the service is deployed in a database cluster consisting of a first database node and a second database node. Users can initiate access requests to the database cluster according to their needs. When access requests to the database cluster are received, they are assigned to the first database node and the second database node for processing according to the domain name server.

[0107] In some examples, the first access request is an access request sent to the second database node. Since the second database node only processes read requests in the business, the first database node is responsible for all write requests and can also process read requests. After the second database node receives the first access request, it can transfer all write requests in the first access request to the first database node for processing through the read-write separation function provided by the database middleware, while the read requests in the first access request are processed by the second database node.

[0108] Step 402: Obtain current server resource usage corresponding to the first database node.

[0109] In some embodiments of the present invention, the current server resource usage includes a current server resource usage rate.

[0110] As some examples, SQL (Structured Query Language) can be used to query the current server resource usage (such as central processing unit, input / output, and memory usage) corresponding to the first database node to determine the current server resource usage; wherein the current server resource usage is used to determine whether to migrate access requests and whether to modify the read request allocation ratio.

[0111] Step 403: Obtain the current access request frequency of the first database node.

[0112] As some examples, the structured query language can be used to query the current access request frequency of the first database node or the second database node; by using the structured query language, the execution times of various types of commands since the first database node or the second database node was started can be queried, including insert, delete, update, and select, with insert, delete, and select as writes and select as reads; specifically, the query can be performed twice at an interval of 1 minute, and the difference can be used to know how many write requests and how many read requests there are in this 1 minute. By performing a query operation every 1 minute, the current access request frequency of the first database node or the second database node can be obtained.

[0113] Step 404, obtaining a test result of the first database node; wherein the test result includes a correspondence between server resource usage and database read and write frequency.

[0114] As some examples, after purchasing server resources, performance stress testing operations can be performed on the first database node and the second database node, that is, the databases under the first database node and the second database node are read and written uninterruptedly at different frequencies through tools, while monitoring various indicators of the servers where the first database node and the second database node services are located (such as: CPU, IO, memory, etc.), and testing resource utilization from 50% to 100% (increases at intervals of 5%) to see what database read and write frequencies can be supported respectively, thereby determining the test results of the first database node and the second database node, and recording them in a test table for subsequent use.

[0115] The test table is used to determine the consumption of server resources (CPU, IO, memory, etc.) by different read and write conditions of the database; test conditions include: read-only, read (80%) + write (20%), read (50%) + write (50%), write-only, etc. Figure 2 In the figure, the database of cloud A (the first database node) can support a maximum of 70% reads and writes, and the database of cloud B (the second database node) can support a maximum of 30% reads. These figures are determined by test tables. At the same time, the read and write of the database consume different server resources. Generally, there are more writes than reads. When the write requests in subsequent access requests increase, in order to ensure the stability of the first database node, when only 5% of the write requests of the second database node migrate to the first database node, the first database node can send more than 5% of the read requests to the second database node, thereby reducing the consumption of resources of the first database node. The specific meaning of "more than 5%" here needs to be determined by the test table.

[0116] Step 405 : Determine the read request information to be migrated according to the current server resource usage rate, the current access request frequency, and the correspondence between the server resource usage and the database read and write frequency.

[0117] In some examples, the read request information to be migrated may be determined based on the ratio of write requests migrated from the second database node to the first database node; Figure 2 , the proportion of write requests migrated from the second database node to the first database node is 15%, and the proportion of read request information to be migrated may be greater than or equal to 15%.

[0118] Step 406: modify the read request allocation ratio of the first database node and the second database node according to the read request information to be migrated.

[0119] In some examples, the read request information to be migrated may also be the number of read requests; specifically, when the server resource utilization rate is 85%, the access request frequency that the server can withstand is 1000, and the current server resource utilization rate corresponding to the first database node is 85%, and the current access request frequency is 1200, and the extra 200 access requests are the read request information to be migrated, and the 200 read requests can be migrated to the first database node. According to the read request information to be migrated, the operation and maintenance personnel can modify the preset read request allocation ratio of the first database node and the second database node through Zookeeper (coordination service software), such as reducing the read ratio of the first database node and increasing the read ratio of the second database node, so that the master node reduces read requests and the slave node increases read requests, so as to achieve load balancing for the first database node.

[0120] For example, if the first database node is DB1, the second database node is DB2, and the preset read request allocation ratio is set to DB1:DB2=3:7, when the read request of the first database node needs to be reduced, the operation and maintenance personnel can modify the configuration to DB1:DB2=2:8.

[0121] Step 407: Migrate the read requests in the second access requests allocated to the first database node to the second database node according to the modified read request allocation ratio.

[0122] As some examples, operation and maintenance personnel can pre-set the read request allocation ratio for the first database node and the second database node through Zookeeper (coordination service software) introduced by the software package file, and migrate the read request in the second access request allocated to the first database node to the second database node through the read request allocation ratio. For example, if the first database node is DB1 and the second database node is DB2, the read request allocation ratio can be set to DB1:DB2=3:7.

[0123] In some examples, after obtaining the current server resource usage corresponding to the first database node, when it is determined that the current server resource usage is greater than the preset server resource usage, the operation and maintenance personnel can modify the pre-set read request allocation ratio for the first database node and the second database node according to the current server resource usage, and migrate the read request in the second access request allocated to the first database node to the second database node according to the modified read request allocation ratio; if the current server resource usage is 85% and the preset server resource usage is 70%, in order to balance the load of the first database node, it is necessary to reduce the read requests of the first database node, and the operation and maintenance personnel can modify the read request allocation ratio to DB1:DB2=2:8.

[0124] As some examples, Zookeeper (coordination service software) can provide a TCP (Transmission Control Protocol) connection method for operation and maintenance personnel to modify the read request distribution ratio.

[0125] As some examples, the preset server resource utilization rate may be 70% or 80%, etc. Those skilled in the art may determine this by performing performance stress testing operations on the first database node and the second database node, and the present invention does not impose any limitation on this.

[0126] In an embodiment of the present invention, by receiving an access request for the database cluster, allocating the access request to the first database node and the second database node, and migrating all write requests in the first access request allocated to the second database node to the first database node; obtaining the current server resource usage corresponding to the first database node; obtaining the current access request frequency of the first database node; obtaining a test result for the first database node; wherein the test result includes a correspondence between the server resource usage and the database read and write frequency; determining read request information to be migrated according to the current server resource usage, the current access request frequency, and the correspondence between the server resource usage and the database read and write frequency; modifying the read request allocation ratio of the first database node and the second database node according to the read request information to be migrated; and migrating the read request in the second access request allocated to the first database node to the second database node according to the modified read request allocation ratio, thereby realizing migration of the read request of the first database node to the second database node according to the current server resource usage, reducing the load of the first database node, and improving the resource utilization rate of the second database node, thereby realizing load balancing of read requests in the database cluster.

[0127] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0128] Reference Figure 5 , shows a schematic diagram of the structure of a load balancing device based on a database cluster provided by an embodiment of the present invention, wherein the database cluster includes a first database node and a second database node, wherein the first database node is used to process write requests and read requests, and the second database node is used to process read requests, and specifically may include the following modules:

[0129] An access request allocation module 501 is used to receive access requests for the database cluster, allocate the access requests to the first database node and the second database node, and migrate all write requests in the first access request allocated to the second database node to the first database node;

[0130] The read request migration module 502 is used to obtain the current server resource usage corresponding to the first database node, and migrate at least part of the second access requests allocated to the first database node to the second database node according to the current server resource usage.

[0131] In some embodiments of the present invention, migrating at least part of the read requests in the second access request allocated to the first database node to the second database node according to the current server resource usage includes:

[0132] According to the current server resource usage, the read request allocation ratio of the first database node and the second database node is modified, and according to the modified read request allocation ratio, the read request in the second access request allocated to the first database node is migrated to the second database node.

[0133] In some embodiments of the present invention, the current server resource usage includes a current server resource usage rate, and modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage includes:

[0134] Obtaining a current access request frequency of the first database node;

[0135] The read request allocation ratio of the first database node and the second database node is modified according to the current server resource usage rate and the current access request frequency.

[0136] In some embodiments of the present invention, modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage rate and the current access request frequency includes:

[0137] Obtaining a test result for the first database node; wherein the test result includes a correspondence between server resource usage and database read and write frequency;

[0138] Determine the read request information to be migrated according to the current server resource usage rate, the current access request frequency, and the correspondence between the server resource usage and the database read and write frequency;

[0139] According to the read request information to be migrated, the read request allocation ratio of the first database node and the second database node is modified.

[0140] In some embodiments of the present invention, the current server resource usage includes a current server resource usage rate;

[0141] Before modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage, the method further includes:

[0142] Determining whether the current server resource usage rate is greater than a preset server resource usage rate;

[0143] When the current server resource usage is greater than a preset server resource usage, modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage is performed.

[0144] In some embodiments of the present invention, allocating the access request to the first database node and the second database node includes:

[0145] The access request is allocated to the first database node and the second database node according to the currently configured access request allocation ratio.

[0146] In some embodiments of the present invention, the device further comprises:

[0147] The access request allocation ratio modification module is used to modify the currently configured access request allocation ratio in response to a ratio modification request.

[0148] In some embodiments of the present invention, the device further comprises:

[0149] The read-write separation module is used to perform read-write separation on the read request and the write request in the allocated access request in the first database node and the second database node.

[0150] In some embodiments of the present invention, the device further comprises:

[0151] A data synchronization module is used to synchronize the data in the first database node to the second database node.

[0152] In some embodiments of the present invention, the first database node and the second database node are database nodes in different cloud platforms.

[0153] In an embodiment of the present invention, by receiving an access request for a database cluster, the access request is allocated to a first database node and a second database node, and all write requests in the first access request allocated to the second database node are migrated to the first database node; the current server resource usage corresponding to the first database node is obtained, and according to the current server resource usage, at least part of the read requests in the second access request allocated to the first database node is migrated to the second database node, thereby achieving migration of the read request of the first database node to the second database node according to the current server resource usage, reducing the load of the first database node, and improving the resource utilization rate of the second database node, thereby achieving load balancing of read requests in the database cluster.

[0154] An embodiment of the present invention further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the load balancing method based on the database cluster as described above is implemented.

[0155] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the load balancing method based on a database cluster as described above is implemented.

[0156] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0157] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0158] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0159] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0163] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0164] The load balancing method, device, equipment and medium based on the database cluster are introduced in detail above. The principles and implementation methods of the present invention are explained in detail using specific examples. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A load balancing method based on a database cluster, characterized in that: The database cluster includes a first database node and a second database node, the first database node is used to process write requests and read requests, and the second database node is used to process read requests. The method includes: receiving an access request for the database cluster, allocating the access request to the first database node and the second database node, and migrating all write requests in the first access request allocated to the second database node to the first database node; A current server resource usage corresponding to the first database node is obtained, and according to the current server resource usage, at least part of the read requests in the second access request allocated to the first database node is migrated to the second database node.

2. The method according to claim 1, characterized in that The step of migrating at least part of the second access requests allocated to the first database node to the second database node according to the current server resource usage includes: According to the current server resource usage, the read request allocation ratio of the first database node and the second database node is modified, and according to the modified read request allocation ratio, the read request in the second access request allocated to the first database node is migrated to the second database node.

3. The method according to claim 2, characterized in that The current server resource usage includes a current server resource usage rate, and modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage includes: Obtaining a current access request frequency of the first database node; The read request allocation ratio of the first database node and the second database node is modified according to the current server resource usage rate and the current access request frequency.

4. The method according to claim 3, characterized in that The modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage rate and the current access request frequency includes: Obtaining a test result for the first database node; wherein the test result includes a correspondence between server resource usage and database read and write frequency; Determine the read request information to be migrated according to the current server resource usage rate, the current access request frequency, and the correspondence between the server resource usage and the database read and write frequency; According to the read request information to be migrated, the read request allocation ratio of the first database node and the second database node is modified.

5. The method according to any one of claims 1 to 4, characterized in that: The current server resource usage includes the current server resource usage rate; Before modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage, the method further includes: Determining whether the current server resource usage rate is greater than a preset server resource usage rate; When the current server resource usage is greater than a preset server resource usage, modifying the read request allocation ratio of the first database node and the second database node according to the current server resource usage is performed.

6. The method according to any one of claims 1 to 4, characterized in that: The allocating the access request to the first database node and the second database node comprises: The access request is allocated to the first database node and the second database node according to the currently configured access request allocation ratio.

7. The method according to claim 6, characterized in that Also includes: In response to the ratio modification request, the currently configured access request allocation ratio is modified.

8. The method according to any one of claims 1 to 4, characterized in that: After allocating the access request to the first database node and the second database node, the method further includes: In the first database node and the second database node, read-write separation is performed on the read request and the write request in the allocated access request.

9. The method according to any one of claims 1 to 4, characterized in that: Also includes: Synchronize the data in the first database node to the second database node.

10. The method according to any one of claims 1 to 4, characterized in that: The first database node and the second database node are database nodes in different cloud platforms.

11. A load balancing device based on a database cluster, characterized in that: The database cluster includes a first database node and a second database node, the first database node is used to process write requests and read requests, and the second database node is used to process read requests. The device includes: an access request allocation module, configured to receive access requests for the database cluster, allocate the access requests to the first database node and the second database node, and migrate all write requests in the first access request allocated to the second database node to the first database node; A read request migration module is used to obtain the current server resource usage corresponding to the first database node, and according to the current server resource usage, migrate at least part of the second access requests allocated to the first database node to the second database node.

12. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 10 when executed by the processor.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images