Database disaster tolerance synchronization method based on packet replication and database system

Abstract

The invention relates to the field of databases, in particular to a database disaster tolerance synchronization method based on packet replication and a database system. The invention provides a refined synchronization control mechanism based on a grouping strategy aiming at the problem that a unified global synchronization strategy cannot meet refined and differentiated requirements of distributed applications on data consistency, availability and performance in the existing database replication technology. Multiple groups of standby libraries adopt different synchronization levels to perform grouping copying in the same database; a logic abstraction layer of a synchronization group is introduced, a synchronization strategy and a physical standby library are decoupled, and multiple different synchronization mechanisms of a master-slave replication process are allowed in a database; a synchronous waiting mode of a traditional single path is broken through by utilizing a multi-waiting queue and a grouping ACK processing mechanism, so that state advancement of different groups does not interfere with each other; and checking is completed based on the consistency of the state mask, so that the transaction can be successfully returned only after all specified synchronization conditions are met, and the strong consistency of data is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of databases, in particular, to a database disaster recovery synchronization method based on group replication and a database system. BACKGROUND

[0002] With the deepening of digital transformation, modern application systems have made unprecedentedly high requirements on the availability, reliability and performance of databases. Under this background, the disaster recovery and read-write separation architecture based on master-slave replication has become the standard configuration of database systems. The core goal of this architecture is to ensure data security, achieve rapid failure recovery, and improve overall read performance. However, the diversification of business scenarios has led to the refinement of data consistency requirements. For example, core payment transactions require strong consistency of data (such as RPO≈0), while user behavior analysis or log processing scenarios can accept second-level or even minute-level eventual consistency. This differentiated demand for data consistency levels by different businesses within the same system has become a key challenge that the current database field must face.

[0003] Existing database master-slave replication technologies mainly include asynchronous replication, semi-synchronous replication and synchronous replication modes. Although these technologies are mature and widely used, they have a fundamental flaw: their replication strategy is a global, rigid "one-size-fits-all" mode. Specifically, administrators can only choose a unified synchronization mode for the entire database instance. This coarse-grained control mechanism raises three significant problems: 1. Architecture rigidity, difficult to balance performance and consistency: If a strong synchronization mode is enabled to ensure data security, the delay of all write operations will be determined by the standby library with the furthest geographical location and the worst performance. This results in sacrificing the write performance of the entire system for the strong consistency requirements of a small number of critical businesses. Conversely, if an asynchronous mode is used to ensure performance, all data (including critical data) is at risk of loss, which cannot meet the reliability requirements of core businesses.

[0004] 2. Low resource utilization: In cross-regional deployment, the hardware capabilities and network delays of different standby libraries differ greatly. Placing high-performance in-room standby libraries and high-delay off-site disaster recovery libraries in the same synchronization group cannot assign appropriate responsibilities to standby libraries according to their actual capabilities, resulting in resource waste.

[0005] 3. High operational complexity: In order to accommodate different needs, architects are often forced to deploy multiple independent database clusters or introduce complex middleware layers to simulate differentiated synchronization. This not only significantly increases hardware and operational costs, but also dramatically increases the overall complexity of the system, challenging its stability.

[0006] In summary, the current database field urgently needs an innovative solution that can break through the limitations of existing replication patterns and implement fine and customizable synchronization strategies within a single database instance to truly achieve the best balance between data consistency, system performance, and resource costs. SUMMARY

[0007] The application aims to provide a database system compatible with multiple databases and realizing zero code modification in the migration process, and an application method thereof.

[0008] According to a first aspect of the application, a database disaster recovery synchronization method based on group replication is proposed, applied to a master database, comprising: S1, configuring a plurality of synchronization groups, each synchronization group including an independently defined member slave database identification list and a group synchronization level; S2, in response to a transaction submission request, determining one or more target synchronization groups associated with the transaction; S3, continuously receiving confirmation messages from a plurality of slave databases, and determining whether each target synchronization group meets the requirements of the corresponding group synchronization level according to the confirmation messages; S4, in the case where all target synchronization groups meet the requirements of the group synchronization level, completing the transaction submission and returning success to the client.

[0009] According to some embodiments, in the method of the first aspect of the application, step S3 is implemented through a group completion state mask mechanism, including: maintaining a group of completion state masks and a target synchronization group mask for the transaction process; wherein the target synchronization group mask is used to identify the target synchronization groups associated with the transaction, and the group completion state mask is used to identify whether the synchronization requirements of each target synchronization group are met; listening to confirmation messages from a plurality of slave databases, the confirmation messages including slave database identification, persisted log sequence number LSN, and operation type corresponding to the log sequence number LSN; determining whether the target synchronization group meets the synchronization requirements according to the target synchronization group to which the confirmation message belongs according to the slave database identification, and according to the operation type and the group synchronization level of the target synchronization group; in the case of meeting, updating the group completion state mask; checking whether the group completion state mask matches the target synchronization group mask to determine whether the synchronization requirements of all target synchronization groups have been met.

[0010] According to some embodiments, in the method of the first aspect of the application, the group completion state mask is located through a plurality of waiting queues, specifically including: maintaining an independent waiting queue for each synchronization group; adding the process identification information and the corresponding log sequence number LSN of the transaction to the waiting queue of the target synchronization group; locating the process that needs to update the group completion state mask by traversing the entries in the waiting queue and comparing the log sequence number LSN in the confirmation message.

[0011] According to some embodiments, in the method of the first aspect of the present application, the group completion status mask and the target synchronization group mask are bit masks, each bit of the bit masks corresponding to a synchronization group.

[0012] According to some embodiments, in the method of the first aspect of the present application, the group synchronization level comprises: remote_write, which requires the state that the log data has been received by the slave and written into the operating system buffer; remote_flush, which requires the state that the log data has been persisted to the storage disk by the slave; and remote_apply, which requires the state that the log data has been redone by the slave and the data change has been applied to its database.

[0013] According to some embodiments, in the method of the first aspect of the present application, the transaction level parameter or the session level parameter is included in the commit request, and the step S2 specifically comprises: dynamically specifying one or more target synchronization groups associated with the transaction according to the transaction level parameter or the session level parameter.

[0014] According to some embodiments, in the method of the first aspect of the present application, the method further comprises a fault handling mechanism, specifically comprising: configuring a timeout time corresponding to the synchronization group; if the synchronization requirement of the synchronization group is still not met within the timeout time, executing a predefined degradation strategy; the degradation strategy comprises: skipping the waiting for the current synchronization group and completing the commit of the transaction, and generating an alarm at the same time; or making the commit of the transaction fail and returning an error.

[0015] According to the second aspect of the present application, a database disaster recovery synchronization method based on group replication is provided, which is applied to a slave, and comprises: registering a unique slave identifier to a master; receiving a write-ahead log (WAL) data stream from the master; performing persistent processing on the received WAL data stream according to the configuration and processing capacity of the current slave; and sending an acknowledgement message to the master after the processing of the WAL data stream reaches one or more phase states, the acknowledgement message comprising the slave identifier, a log sequence number (LSN) that has been persisted, and an operation type corresponding to the log sequence number (LSN).

[0016] According to some embodiments, in the method of the second aspect of the present application, the operation type comprises receiving write, flushing, and applying.

[0017] According to the third aspect of the present application, a database system is provided, comprising: a master configured to execute the method of the first aspect of the present application; and a plurality of slaves configured to execute the method of the second aspect of the present application.

[0018] This invention addresses the problems of rigid structure, low resource utilization, inadequate matching of business requirements, and high operational complexity in existing database systems by introducing logical abstraction of synchronization groups, state masks, multiple wait queue mechanisms, and transaction-level synchronization target specification. It offers the following beneficial effects: 1. Balance between performance and consistency: The system can simultaneously provide financial-grade strong data consistency (requires remote_apply for near-end groups) and high-performance disaster recovery in remote locations (requires remote_write for remote groups), so that write performance is no longer constrained by the slowest remote backup database.

[0019] 2. Architectural flexibility and resource optimization: It can easily adapt to complex deployment topologies of multiple data centers, cross-regional deployments, and hybrid clouds. A powerful standby database can handle the heavy responsibility of strong consistency reads and failover, while a weaker standby database or a remote standby database focuses on data backup and off-site disaster recovery, making the best use of resources and maximizing resource utilization.

[0020] 3. Business-aware adaptive data persistence: Applications can dynamically specify the data persistence requirements of a transaction through transaction-level parameters. This means that business logic can directly drive the underlying data consistency level, achieving true business and infrastructure matching.

[0021] 4. Simplified Operation and High Availability: Existing solutions often require the deployment of multiple independent clusters to meet different consistency requirements. This invention supports multiple synchronization strategies within a single database instance, requiring only one database cluster to achieve the disaster recovery goals that previously required multiple clusters. The failover strategy becomes clearer, prioritizing the backup database that is closest to the primary database and belongs to the strong synchronization group, thereby ensuring that the RPO (Recovery Point Objective) is minimized. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without exceeding the scope of protection claimed by the present invention.

[0023] Figure 1 This is a flowchart illustrating an embodiment 1000 of the database disaster recovery synchronization method based on group replication of the present invention applied to a primary database. Figure 2 for Figure 1 A flowchart illustrating the state masking mechanism S3A in step S3 of embodiment 1000; Figure 3 This is a flowchart illustrating an embodiment 2000 of the database disaster recovery synchronization method based on group replication of the present invention applied to a slave database. Figure 4 This is a schematic diagram of the structure of embodiment 3000 of the database system of the present invention; Figure 5 for Figure 4 The following is a time-series diagram illustrating the dynamic interaction relationships of the database system during the transaction commit process, as shown in Example 3000. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Figure 1 This is a flowchart illustrating an embodiment 1000 of the database disaster recovery synchronization method based on group replication of the present invention applied to a primary database. Figure 1 As shown, Example 1000 includes steps S1-S4.

[0026] In step S1, the master database is configured with multiple synchronization groups. Each synchronization group includes an independently defined list of member slave database identifiers and a group synchronization level.

[0027] Optionally, a synchronization group's identifier list includes the slave identifiers of all its member slaves, serving as unique identifiers for slaves within the current database system. Optionally, group synchronization levels include: remote_write, which requires the slave to have received log data and written it to the operating system buffer; remote_flush, which requires the slave to have persisted the log data to the storage disk; and remote_apply, which requires the slave to have redoed the log data and applied the data changes to its database.

[0028] In some specific implementations, in enterprise-level database application scenarios, a primary production center and several local disaster recovery centers are typically configured, with the local disaster recovery centers and the primary production center geographically separated. For the primary production center: it is typically configured with one master database and multiple slave databases to handle business load. The multiple data nodes in the primary production center must be in a strongly synchronized state. If the master database encounters a problem, a slave database can be promoted to master node through an election process to ensure the smooth completion of the main business. To the same city disaster recovery center: one or more slaves can be configured for disaster recovery of database key data. Multiple slaves of the same city disaster recovery center do not need to undertake real-time data consistency of the application, but only need to implement data backup and data redundancy to ensure that data is not lost.

[0029] Step S1 can implement the slave of different data synchronization requirements, and define the group synchronization level respectively. For example, in some embodiments, the database system includes a master library of a main production center and a slave 1, a slave 2, and a slave 3 of a same city disaster recovery center. In step S1, the master library is configured: synchronization group 1 includes slave 1 and slave 2 of the main production center, and the group synchronization level is remote_flush; synchronization group 2 includes slave 3 of the same city disaster recovery center, and the group synchronization level is remote_write.

[0030] In step S2, in response to the commit request of the transaction, the master library determines one or more target synchronization groups related to the transaction. Optionally, the transaction level parameter or the session level parameter is included in the commit request, which is used to determine which target synchronization group the transaction needs to wait for the synchronization confirmation message. The application program can dynamically specify one or more target synchronization groups that the transaction needs to wait for in the commit request through the transaction level parameter or the session level parameter, so that the business logic (such as the transaction type) can directly and explicitly drive the underlying data synchronization strength without modifying the global configuration of the database or relying on external middleware. For example, the payment business can specify to wait for the remote_apply group, and the user behavior analysis business can specify to wait for only the remote_write group, thereby realizing the precise matching of business requirements and data consistency level.

[0031] Optionally, the process of step S2 performed by the master library specifically includes: dynamically specifying one or more target synchronization groups associated with the transaction according to the transaction level parameter or the session level parameter.

[0032] In step S3, the master library continuously receives the confirmation messages of the multiple slaves, and judges whether each target synchronization group meets the requirements of the corresponding group synchronization level according to the confirmation messages. Optionally, step S3 is implemented through a state mask mechanism and multiple waiting queues, which will be described in detail in Figure 2 .

[0033] In step S4, in the case where all target synchronization groups meet the requirements of the group synchronization level, the commit of the transaction is completed and the success is returned to the client. Optionally, in step S4, the group synchronization levels of all target synchronization groups are met, at which time the back-end process is woken up, and the commit process of the transaction is finally completed.

[0034] In some embodiments, the method performed by the master further comprises a failure handling mechanism, which comprises: configuring a timeout for the synchronization group; if the synchronization requirement of the synchronization group is not satisfied within the timeout, executing a predefined degradation strategy; the degradation strategy comprises: skipping the waiting for the current synchronization group and completing the transaction submission, while generating an alarm; or, failing the transaction submission and returning an error.

[0035] Figure 2 For Figure 1 a flowchart of the state mask mechanism S3A in step S3 in embodiment 1000. As shown in Figure 2 , the state mask mechanism S3A comprises steps S31-S34.

[0036] Optionally, in step S3, the group completion state mask is located through a multi-wait queue, which comprises: maintaining an independent wait queue for each synchronization group; adding the process identification information of the transaction and the corresponding log sequence number LSN to the wait queue of the target synchronization group; locating the process that needs to update the group completion state mask by traversing the entries in the wait queue and comparing the log sequence number LSN in the confirmation message.

[0037] In step S31, the master maintains a group completion state mask and a target synchronization group mask for the process of the transaction; the target synchronization group mask is used to identify the target synchronization group associated with the transaction, and the group completion state mask is used to identify whether the synchronization requirement of each target synchronization group is satisfied. Optionally, the group completion state mask and the target synchronization group mask are bit masks, each bit of the bit mask corresponds to a synchronization group. For example, the target synchronization group of transaction T1 is group 1, then the target synchronization group mask syncRepGroupMask corresponding to transaction T1 is 0b01, and the group completion state mask syncRepGroupDone is initialized to 0b00.

[0038] In step S32, the master listens to the confirmation messages from multiple slaves, which include the slave identification, the persisted log sequence number LSN, and the operation type corresponding to the log sequence number LSN. Optionally, the operation type includes receiving write, flushing, and applying. For example, one specific embodiment of the confirmation message ACK is: {application_name:'standby3', flushed_lsn: LSN101, applied_lsn: LSN101}.

[0039] At step S33, the master database obtains the target synchronization group to which the acknowledgement message belongs according to the slave database identifier, and judges whether the target synchronization group meets the synchronization requirement according to the operation type and the group synchronization level of the target synchronization group; in the case of meeting, the group completion state mask is updated. Optionally, the master database 1 obtains the target synchronization group and the group synchronization level to which the acknowledgement message belongs according to the slave database identifier, and locates the corresponding waiting queue in combination with the member slave database identifier list in the plurality of synchronization groups configured by itself.

[0040] For example, one specific embodiment of step S33 includes: receiving an ACK acknowledgement message, checking that the acknowledgement message is from the slave database 1, the slave database 1 belongs to the synchronization group 1, and the group synchronization level of the synchronization group 1 is remote flush; the first log sequence number LSN of the flush operation in the ACK acknowledgement message is 101, the second log sequence number LSN of the flush operation in the waiting queue of the synchronization group 1 is 101, and the states of all member slave databases in the synchronization group 1 are confirmed to be completed by comparing the two LSNs. Then, the group completion state mask of the synchronization group 1 is updated.

[0041] At step S34, the master database checks whether the group completion state mask matches the target synchronization group mask to judge whether the synchronization requirements of all target synchronization groups have been met. The master database uses the bitwise comparison of the group completion state mask and the target synchronization group mask to realize the atomic strong consistency of the synchronization requirements of the target synchronization group at step S34.

[0042] Figure 3 The flowchart of the embodiment 2000 of the application of the database disaster recovery synchronization method based on group replication to the slave database.

[0043] As shown in Figure 3 The embodiment 2000 includes steps S201-S204. At step S201, the slave database registers a unique slave database identifier with the master database. At step S202, the slave database receives the WAL data stream from the master database. At step S203, the slave database performs the persistent processing on the received WAL data stream according to the current configuration and processing capacity of the slave database. At step S204, after the processing of the WAL data stream reaches one or more phase states, the slave database sends an acknowledgement message to the master database, and the acknowledgement message includes the slave database identifier, the persistent log sequence number LSN, and the operation type corresponding to the log sequence number LSN. Optionally, the operation type includes receiving writing, flushing, and applying.

[0044] There is a slave database with a slave database identifier of standby_1, and the core task of the slave database is to participate in a synchronization group with a synchronization requirement of remote apply, as shown in Figure 3The implementation process of the slave standby_1 in the embodiment 2000 shown includes: Step S201: When the WalReceiver process of the slave establishes a connection with the WalSender process of the master, it transmits application_name='standby_1'. The master records this identifier and matches it with the member list of the synchronization group.

[0045] Step S202: The WalReceiver process of the slave continuously receives the WAL data stream from the WalSender process of the master. For example, transaction T1 is committed on the master, generating a WAL record with LSN 0 / 18000060. The WalSender of the master sends these binary log data to standby_1 through the network.

[0046] Step S203: After the slave receives the WAL data stream, the persistence process is carried out in stages, including: Stage one, receiving and writing to the operating system buffer: the WalReceiver process writes the received WAL data to the WAL segment file of the slave. At this time, the data is in the page cache of the operating system and has not been physically written to the disk. This stage corresponds to the remote_write level requirement. For a synchronization group with a group synchronization level of remote_write, the slave can send an ACK after completing this stage.

[0047] Stage two, disk persistence: the slave periodically (or according to the configuration) calls fsync() or similar system calls to force the WAL data in the operating system buffer to be flushed to the persistent storage device (such as an SSD hard disk). This stage corresponds to the requirement of the remote_flush synchronization level.

[0048] Stage three, application data change: the startup process of the slave reads the persisted WAL record, parses the data change (such as INSERT, UPDATE) therein, and applies it to the data table of the slave. This stage corresponds to the requirement of the remote_apply synchronization level. This is the most stringent requirement, meaning that the data of the slave is logically consistent with the master and can provide a no-delay read service.

[0049] Step S204: When the processing reaches the stage state, the slave sends an acknowledgement message to the master. The acknowledgement message includes: application_name: the unique identifier of the slave, letting the master know that the ACK comes from whom; LSN: the persisted log sequence number, informing the master that "the processing has reached this position"; state / operation type: operation type, explicitly tells the master library the persistence stage (write / flush / apply) reached by this LSN.

[0050] Figure 4 For the structural schematic of the embodiment 3000 of the database system of the application, Figure 5 For Figure 4 The timing schematic of the dynamic interaction relationship of the database system shown in the embodiment 3000 in the commit process of the transaction T1.

[0051] As Figure 4 shown, the scenario configuration of the embodiment 3000 includes: The database system includes a master library, a slave library 1, a slave library 2, and a slave library 3; the master library and the slave library 1 and the slave library 2 are located in the main production center, and the slave library 3 is located in the same city disaster recovery center.

[0052] Configure synchronization groups: Synchronization group 1: slave library 1, application name standby1; slave library 2, application name standby2; group synchronization level remote flush; Synchronization group 2: slave library 1, application name standby1; slave library 3, application name standby3; group synchronization level remote apply.

[0053] Transaction: the target synchronization groups of the transaction T1 are the synchronization group 1 and the synchronization group 2; the target synchronization group of the transaction T2 is the synchronization group 1.

[0054] Master library internal state initialization: Maintain a separate waiting queue SyncRepQueue for the synchronization group 1 and the synchronization group 2 respectively; The transaction T1 and the transaction T2 are respectively configured with a group completion state mask syncRepGroupDone for tracking the completion of each synchronization group, and the mask bit 0 represents group 1 and the bit 1 represents group 2; Master library configuration parameters: synchronous_standby_groups = 'group1, group2' synchronous_standby_names.group1 ='standby1, standby2' synchronous_commit.group1 ='remote_flush' synchronous_standby_names.group2 ='standby1, standby3' synchronous_commit.group2 ='remote_apply' As shown in Figure 5 the commit process of transaction T1 includes: 1. Transaction T1 commit preparation Assign a log sequence number LSN101 to T1; The target synchronous groups of T1 are synchronous group 1 and synchronous group 2, and the target synchronous group mask of T1 is calculated as syncRepGroupMask 0b11; Add the PID and LSN101 of T1 to the waiting queue of synchronous group 1 and synchronous group 2; The group completion status mask of T1 is initialized to 0b00.

[0055] 2. Log sending and ACK processing The master sends the log data of LSN101 to all slaves; Slave 1 and slave 2 (members of synchronous group 1): After receiving the log, immediately write the log to disk (reach the remote flush level); Slave 1 sends an ACK to the master: {application_name:'standby1', flushed_lsn: LSN101, applied_lsn: LSN100}; Slave 2 sends an ACK to the master: {application_name:'standby2', flushed_lsn: LSN101, applied_lsn: LSN100}; Slave 3 (group 2 member): After receiving the log, write it to disk, and then start applying the log, which takes time to reach the remote_apply level.

[0056] 3. Master checks synchronous group 1 completion After receiving the ACK from slave 1, the master checks that slave 1 belongs to synchronous group 1 and synchronous group 2.

[0057] For synchronous group 1, flushed_lsn ≥ LSN101, mark the status of slave 1 in synchronous group 1 as complete; similarly, after receiving the ACK from slave 2, the master marks the status of slave 2 in synchronous group 1 as complete.

[0058] Sync group 1 requires all members (slave 1 and slave 2) to reach remote_flush, so the sync group 1 group sync requirement has been met.

[0059] Master updates T1's group done status mask: set bit 0 (group 1) to 1, now syncRepGroupDone = 0b01, but syncRepGroupDone (0b01) is not equal to syncRepGroupMask (0b11), so T1 continues to wait.

[0060] 4. Master checks sync group 2 completion Slave 1: after applying the log of LSN 101, the application LSN reaches LSN 101, sends a new ACK to the master: {application_name:'standby1', flushed_lsn: LSN101, applied_lsn: LSN101}.

[0061] After receiving, the master checks that slave 1 belongs to sync group 2 and applied_lsn ≥ LSN101, so marks the status of slave 1 in group 2 as complete.

[0062] Slave 3: after applying the log of LSN 101, sends an ACK to the master: {application_name:'standby3', flushed_lsn: LSN101, applied_lsn: LSN101}.

[0063] After receiving, the master marks the status of slave 3 in sync group 2 as complete.

[0064] Sync group 2 requires all members (slave 1 and slave 3) to reach remote_apply, so the sync group 2 sync requirement has been met.

[0065] Master updates T1's group done status mask: set bit 1 (group 2) to 1, now syncRepGroupDone = 0b11.

[0066] 5. Transaction T1 completes commit Master compares syncRepGroupDone and syncRepGroupMask (both are 0b11), the condition is met.

[0067] Master wakes up the process of T1, returns commit success to the client, and removes T1's process from the waiting queue of sync group 1 and sync group 2.

[0068] Optionally, the transaction T2 also verifies whether the slave database 1 and the slave database 2 in the related synchronization group 1 meet the synchronization requirement of the remote flush through the above process, and updates the group completion state mask syncRepGroupDone of T2 to 0b01 in the case of meeting the requirement. The syncRepGroupDone and the syncRepGroupMask (both are 0b01) are compared, and the condition is met. The transaction T2 completes the submission: the master database wakes up the process of T2, and returns the submission success to the client; and the process of T2 is removed from the waiting queue of the group 1.

[0069] By classifying the transaction granularity, the performance and consistency balance of finer granularity are realized, the performance bottleneck of the traditional synchronization mode is broken, and the system can require the remote_flush level for the synchronization group 1 and the remote_apply level for the synchronization group 2 in the same transaction submission process. This means that the key transaction (such as payment) can obtain the financial level strong consistency (RPO≈0) by waiting for the strong synchronization group (such as group 2), and the non-key transaction (such as log recording) only needs to wait for the weak synchronization group (such as group 1) to quickly return. This mechanism ensures that the write performance is no longer determined by the standby database with the farthest geographical position or the worst processing capacity (such as the slave database 3 in the embodiment), but by the worst performance member in the target synchronization group actually specified by the transaction, so that the overall write throughput of the system is significantly improved on the premise of ensuring the safety of core business data.

[0070] The above describes the embodiments of the present application in detail, and the principle and implementation mode of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method of the present application and its core idea. Meanwhile, the changes or deformations made by the person skilled in the art according to the principle of the present application, based on the specific implementation mode and application range of the present application, all belong to the protection range of the present application. In summary, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A database disaster recovery synchronization method based on packet replication, applied to a master database, characterized in that, The method comprises: S1, configuring a plurality of synchronization groups, each of which comprises an independently defined identification list of member slave databases and a group synchronization level; S2, in response to a commit request of a transaction, determining one or more target synchronization groups associated with the transaction; S3, continuously receiving acknowledgement messages of a plurality of slave databases, and determining whether each of the target synchronization groups meets the requirement of the corresponding group synchronization level according to the acknowledgement messages; S4, in the case where all the target synchronization groups meet the requirement of the group synchronization level, completing the commit of the transaction and returning success to the client.

2. The method of claim 1, wherein, The step S3 is implemented through a group completion state mask mechanism, comprising: maintaining a group of completion state masks and a target synchronization group mask for the process of the transaction; wherein the target synchronization group mask is used to identify the target synchronization groups associated with the transaction, and the group completion state mask is used to identify whether the synchronization requirement of each of the target synchronization groups is met; listening to the acknowledgement messages from the plurality of slave databases, wherein the acknowledgement messages comprise slave database identification, a persisted log sequence number LSN, and an operation type corresponding to the log sequence number LSN; obtaining the target synchronization group to which the acknowledgement message belongs according to the slave database identification, and determining whether the target synchronization group meets the synchronization requirement according to the operation type and the group synchronization level of the target synchronization group; in the case of meeting, updating the group completion state mask; checking whether the group completion state mask matches the target synchronization group mask to determine whether the synchronization requirement of all the target synchronization groups has been met.

3. The method of claim 2, wherein, The group completion state mask is completed through a plurality of waiting queues, specifically comprising: maintaining an independent waiting queue for each of the synchronization groups; adding the process identification information of the transaction and the corresponding log sequence number LSN to the waiting queue of the target synchronization group; locating the process which needs to update the group completion state mask by traversing the entries in the waiting queue and comparing the log sequence number LSN in the acknowledgement message.

4. The method according to claim 2 or 3, characterized in that, The group completion state mask and the target synchronization group mask are bit masks, and each bit of the bit mask corresponds to one of the synchronization groups.

5. The method of claim 1, wherein, The group synchronization level comprises: remote write remote_write, which requires a state that the slave database has received log data and written the operation system buffer; remote flush remote_flush, which requires a state that the slave database has persisted the log data to the storage disk; remote apply remote_apply, which requires a state that the slave database has redone the log data and applied the data change to its database.

6. The method of claim 1, wherein, The commit request comprises a transaction-level parameter or a session-level parameter, and the step S2 specifically comprises: dynamically specifying one or more target synchronization groups associated with the transaction according to the transaction-level parameter or the session-level parameter.

7. The method of claim 1, wherein, Further comprising a failure processing mechanism, specifically comprising: configuring a timeout time corresponding to the synchronization group; If the synchronization requirement of the synchronization group is still not met within the timeout time, a predefined degradation strategy is executed; the degradation strategy includes: skipping the waiting for the current synchronization group and completing the commit of the transaction, while generating an alarm; or, failing the commit of the transaction and returning an error.

8. A database disaster recovery synchronization method based on packet replication, applied to a slave, characterized in that, The method comprises: registering a unique slave library identifier to a master library; receiving a write-ahead log (WAL) data stream from the master library; persisting the received WAL data stream according to the configuration and processing capacity of the current slave library; after the processing of the WAL data stream reaches one or more phase states, sending a confirmation message to the master library, wherein the confirmation message includes a slave library identifier, a persisted log sequence number (LSN), and an operation type corresponding to the log sequence number (LSN).

9. The method of claim 8, wherein, The operation type includes receiving a write, disk flushing, and application.

10. A database system, characterized in that The method comprises: a master library for executing the method according to any one of claims 1-7; a plurality of slave libraries for executing the method according to claim 8 or 9.

Images