A database cluster master-slave method and device

Abstract

The present invention relates to the field of databases, and specifically provides a database cluster master-slave method, which has the following steps: S1, after opening a transaction, a unique transaction ID is obtained and returned to the client; S2, the user sends an SQL sql1 to the connection node, and the connection node forwards the received sql1 to the corresponding coordinator node for execution, responds to the connection node after the execution is completed and returns to the client, and at the same time sends the SQL statement or the Key-Value data corresponding to the SQL statement and the dependency relationship between transactions to the Merge module of the backup site; S3, the user sends a second SQL sql1, and the process is the same as the above step S2; S4, the user sends a commit / abort statement, and the transaction enters the commit stage; S5, after receiving the commit, the Merge module of the backup site verifies whether the transaction is complete through the key span recorded in the transaction and the last version number of each range. Compared with the prior art, the present invention can quickly switch to the backup database after the main database fails, thereby improving the disaster tolerance capability of the database.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and specifically provides a database cluster master-slave method and device. Background Art

[0002] A database backup database is a backup database that creates, maintains, and monitors one or more primary databases to protect the enterprise data structure from failures, disasters, errors, and crashes. Currently, database backup technology is being widely used around the world. Since the 9 / 11 incident, many companies have given priority to disaster recovery in different locations. The most valuable thing in the 21st century is data. As a basic IT software for storing data, traditional stand-alone databases are increasingly limited by poor scalability, smooth performance curves, and high investment in the era of massive data. Distributed databases have emerged as the times require. Distributed databases break the performance bottleneck of traditional stand-alone databases, can be deployed on relatively cheap machines, and theoretically have linear scalability, completely eliminating users' concerns that databases cannot support the rapid development of businesses.

[0003] Distributed databases use computer networks to connect multiple computing and storage units dispersed in physical space into a logically integrated database system, which has the characteristics of dispersion in physical space, integrity in logical structure, and autonomy of independent sites. Therefore, the data distribution in distributed databases is transparent, and it is necessary to use distributed consistency algorithms to reach consensus on various data copies, and to allow appropriate redundancy of data to prevent failure of data copies.

[0004] Since distributed databases usually use Paxos or Raft consensus protocols to achieve disaster recovery through multiple copies of data in a single database cluster, the cluster itself has high availability when a node fails. However, considering the implementation principle of the consensus protocol, when deploying clusters in multiple computer rooms in different locations, a single database cluster has the following challenges:

[0005] On the one hand, in order to improve the access performance of the database, data access is usually provided by nodes in the primary site. However, the data in each node in the backup site is usually behind the primary site and is not consistent. Therefore, if the primary site fails and all nodes fail, the backup site cannot take over directly, that is, it does not have disaster recovery capabilities. However, if the data is copied to the backup site in synchronous mode, although it can ensure that the backup site has the ability to take over the service at any time, it faces the problem that as the geographical distance between the primary and backup sites increases, the synchronous replication mode will cause the service performance in the primary site to drop significantly, affecting the business continuity operation capability.

[0006] On the other hand, the model of deploying a single cluster across sites also has the problem of high interdependence between nodes, which may transmit influence to each other and cause damage when failures occur. Summary of the invention

[0007] The present invention aims at the above-mentioned deficiencies of the prior art and provides a highly practical database cluster master-slave method.

[0008] A further technical task of the present invention is to provide a reasonably designed, safe and applicable master-slave device between database clusters.

[0009] The technical solution adopted by the present invention to solve its technical problem is:

[0010] A database cluster master-slave method includes the following steps:

[0011] S1. The user connects to any connection node of the main database through the database client, starts a transaction, obtains a unique transaction ID and returns it to the client;

[0012] S2. The user sends an SQL sql1 to the connection node. The connection node forwards the received sql1 to the corresponding coordinator node for execution. After the execution is completed, the connection node responds to the client and sends the SQL statement or the Key-Value data corresponding to the SQL statement and the dependency relationship between transactions to the Merge module of the standby site.

[0013] S3, the user sends the second SQL sql1, and the process is the same as the above step S2;

[0014] S4. The user sends a commit / abort statement, and the transaction enters the commit phase. The key span corresponding to the range involved in the current transaction and the version number generated by the number of accesses to the range recorded in the previous SQL execution are ready to be sent to the Merge module. At this time, the commit / abort request response phase can be entered;

[0015] S5. After receiving the commit, the Merge module at the standby site verifies whether the transaction is complete by using the key span recorded in the transaction and the final version number of each range.

[0016] Preferably, in step S2, the dependency relationship between transactions includes the number of times the current transaction accesses the current Range and the writeintent found when the SQL execution accesses the corresponding data row.

[0017] Furthermore, when SQL accesses the corresponding data row, the following steps are performed:

[0018] (1) If there is accessible data, access the next row of data;

[0019] (2) If no write intent is found when accessing data, the write intent of the current transaction is written, and then the data to be sent to the merge module is recorded, and the next row is accessed;

[0020] (3) If a write intent is found when accessing data, the status of the write intent is analyzed to see if it is pending. If not, the write intent of the current transaction is written in an overwritten manner after recording the current write intent, and then the data to be sent to the merge module is also recorded, and the next row is accessed.

[0021] (4) If the status of the parsed write intent is pending, the current transaction conflicts with other transactions and the current transaction enters the waiting state.

[0022] Furthermore, in step S3, the user sends a second SQL sql1, which accesses both the coordinator node including Range1 and the common node including Range2.

[0023] Further, in step S4, the phase of responding to the commit / abort request has two modes, namely, asynchronous mode and synchronous mode, and the asynchronous mode directly responds to the commit / abort request;

[0024] The synchronization mode waits for the data to be sent to the Merge module to be sent, and then responds to the commit / abort request. After the response, updateWriteIntent is called to modify the transaction status in WriteIntent.

[0025] Furthermore, in step S5, it is determined whether the verification transaction is complete. If it is incomplete, continue to wait. If the transaction is complete, check the dependency of the transaction to determine whether the transaction can be written to the standby database. If not, wait for the dependent transactions to be completed. If there is no dependency or the dependent transactions have been written to the standby database, the current transaction is written to the standby database. After writing to the standby database, the status that the current transaction has been written to the standby database is recorded, and a write intent to clean up the current transaction can be initiated to the main database.

[0026] Furthermore, if the write intent is present, the record of the current transaction completion recorded by the merge module will be deleted after cleaning. If the intent is found to be overwritten by the transaction, the record of the current transaction completion will be cleaned up when the discovered transaction is completed.

[0027] A database cluster master-slave device, comprising: at least one memory and at least one processor;

[0028] The at least one memory is used to store a machine-readable program;

[0029] The at least one processor is used to call the machine-readable program to execute a master-slave method between database clusters.

[0030] Compared with the prior art, the database cluster master-slave method and device of the present invention have the following outstanding beneficial effects:

[0031] The present invention solves the problems of data loss and business interruption after the main database fails, as well as the high interdependence between nodes in a single database cluster, which may transmit influences to each other and cause damage when a failure occurs. It improves the continuity of database business, enhances the robustness of the database, and enhances the high availability capability.

[0032] When the main database fails, it can quickly switch to the backup database, improving the disaster recovery capability of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Attached Figure 1 It is a flowchart of a master-slave method among database clusters;

[0035] Attached Figure 2 It is a flowchart of SQL accessing data in the master-slave method among database clusters;

[0036] Attached Figure 3 The figure is a schematic diagram of the Update Write Intent flow in the master-slave method among database clusters. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] A best embodiment is given below:

[0039] like Figure 1As shown, in this embodiment, a database cluster master-slave method is used. The master site has the connection node, coordinator node, and common node shown in the figure, and the slave site has the Merge module and the backup database. Taking a cross-node transaction as an example, the database cluster master-slave solution based on the dependency relationship between transactions is described. The specific method is as follows:

[0040] S1. The user connects to any connection node of the main database through the database client, starts a transaction, and obtains a unique transaction ID and returns it to the client.

[0041] S2. The user sends an SQL sql1 to the connection node, and the connection node forwards the received sql1 to the corresponding coordinator node for execution. After the execution is completed, the connection node responds to the client and sends the SQL statement or the Key-Value data corresponding to the SQL statement and the dependency relationship between transactions to the Merge module of the standby site.

[0042] The dependencies between transactions include the number of times the current transaction accesses the current Range and the write intent found when the SQL execution accesses the corresponding data row.

[0043] like Figure 2 As shown, the detailed process of SQL accessing the corresponding data row is as follows:

[0044] (1) If there is accessible data, access the next row of data;

[0045] (2) If no write intent is found when accessing data, the write intent of the current transaction is written, and then the data to be sent to the merge module is recorded, and the next row is accessed;

[0046] (3) If a write intent is found when accessing data, the status of the write intent is analyzed to see if it is pending. If not, the write intent of the current transaction is written in an overwritten manner after recording the current write intent, and then the data to be sent to the merge module is also recorded, and the next row is accessed.

[0047] (4) If the status of the parsed write intent is pending, the current transaction conflicts with other transactions and the current transaction enters the waiting state.

[0048] S3. The user sends the second SQL sql1. The process is the same as the above step S2. The difference is that this SQL needs to be executed in a distributed manner, which shows that the method supports distributed SQL capabilities.

[0049] This SQL accesses both the coordinator node containing Range1 and the common node containing Range2.

[0050] S4. The user sends a commit / abort statement, and the transaction enters the commit phase. The key span corresponding to the range involved in the current transaction and the version number generated by the number of accesses to the range recorded in the previous SQL execution are ready to be sent to the Merge module. At this time, the commit / abort request response phase can be entered;

[0051] like Figure 3 As shown, there are two modes in the commit / abort request response phase, namely asynchronous (RPO>0) and synchronous (RPO=0). The asynchronous mode can directly respond to the commit / abort request at this time; the synchronous mode needs to wait until the data to be sent to the Merge module has been sent before responding to the commit / abort request. After the response, updateWriteIntent is called to modify the transaction status in WriteIntent.

[0052] S5. After receiving the commit, the Merge module at the standby site verifies whether the transaction is complete through the key span recorded in the transaction and the final version number of each range. If it is incomplete, it continues to wait. If the transaction is complete, it checks the dependency of the transaction to determine whether the transaction can be written to the standby database. If not, it waits for the dependent transactions to be completed. If there is no dependency or the dependent transactions have been written to the standby database, the current transaction can be written to the standby database. After writing to the standby database, it records the status that the current transaction has been written to the standby database, and then initiates a write intent to clean up the current transaction to the primary database.

[0053] If all write intents are present, the records of the current transaction completion recorded in the merge module can be deleted after cleaning. If it is found that the intent is overwritten by other transactions, the deletion of the records of the current transaction completion depends on the completion of other transactions found.

[0054] Based on the above method, a database cluster master-slave device in this embodiment includes: at least one memory and at least one processor;

[0055] The at least one memory is used to store a machine-readable program;

[0056] The at least one processor is used to call the machine-readable program to execute a master-slave method between database clusters.

[0057] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any appropriate changes or substitutions made by a person of ordinary skill in the art to a method and device for master-slave between database clusters that conform to the claims of the present invention shall fall within the patent protection scope of the present invention.

[0058] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A database cluster master-slave method, characterized in that: The following steps are involved: S1. The user connects to any connection node of the main database through the database client, starts a transaction, obtains a unique transaction ID and returns it to the client; S2. The user sends an SQL sql1 to the connection node. The connection node forwards the received sql1 to the corresponding coordinator node for execution. After the execution is completed, the connection node responds to the client and sends the SQL statement or the Key-Value data corresponding to the SQL statement and the dependency relationship between transactions to the Merge module of the standby site. The dependencies between transactions include the number of times the current transaction accesses the current Range and the write intent found when the SQL execution accesses the corresponding data row; S3, the user sends the second SQL sql1, and the process is the same as the above step S2; S4. The user sends a commit / abort statement, and the transaction enters the commit phase. The key span corresponding to the range involved in the current transaction and the version number generated by the number of accesses to the range recorded in the previous SQL execution are ready to be sent to the Merge module. At this time, the commit / abort request response phase can be entered; S5: After receiving the commit, the Merge module at the standby site verifies whether the transaction is complete by using the key span recorded in the transaction and the final version number of each range. Determine whether the verification transaction is complete. If it is not complete, continue to wait. If the transaction is complete, check the transaction dependencies to determine whether the transaction can be written to the slave database. If not, wait for the dependent transactions to be completed. If there is no dependency or the dependent transactions have been written to the slave database, the current transaction is written to the slave database. After writing to the slave database, record the status that the current transaction has been written to the slave database, and then initiate a write intent to clean up the current transaction to the primary database.

2. A database cluster master-slave method according to claim 1, characterized in that: When SQL accesses the corresponding data row, the following steps are performed: (1) If there is accessible data, access the next row of data; (2) If no write intent is found when accessing data, the write intent of the current transaction is written, and then the data to be sent to the merge module is recorded, and the next row is accessed; (3) If a write intent is found when accessing data, the status of the write intent is analyzed to see if it is pending. If not, the write intent of the current transaction is written in an overwritten manner after recording the current write intent, and then the data to be sent to the merge module is also recorded, and the next row is accessed. (4) If the status of the parsed write intent is pending, the current transaction conflicts with other transactions and the current transaction enters the waiting state.

3. A database cluster master-slave method according to claim 2, characterized in that: In step S3, the user sends a second SQL sql1, which accesses both the coordinator node containing Range1 and the common node containing Range2.

4. A database cluster master-slave method according to claim 3, characterized in that: In step S4, the stage of responding to the commit / abort request has two modes, namely, asynchronous mode and synchronous mode, and the asynchronous mode directly responds to the commit / abort request; The synchronization mode waits for the data to be sent to the Merge module to be sent, and then responds to the commit / abort request. After the response, updateWriteIntent is called to modify the transaction status in WriteIntent.

5. A database cluster master-slave method according to claim 4, characterized in that: If all write intents are present, the records of the current transaction completion recorded in the merge module will be deleted after cleaning. If it is found that the intent is overwritten by the transaction, the records of the current transaction completion will be cleaned up when the discovered transaction is completed.

6. A database cluster master-slave device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 5.

Images