A database replication method and system

Abstract

The application provides a database replication method and system, the method comprises the following steps: responding to a binary change log and performing analysis, extracting an initial log source sequence number, a timestamp and an associated partition identifier, and then obtaining a database partition preset sequence number; if the two sequence numbers are not continuous, the timestamp is valid, and the initial log source sequence number satisfies a preset first condition, a new independent numerical range is created and a target missing sequence number interval is identified; a data generation log supplement signal is generated based on the above data and sent to a producer cluster, so that the producer cluster feeds back a missing sequence log; based on the independent numerical range and the database partition preset sequence number, a database partition first sequence number is obtained, and the replication of the database is completed. The database replication method provided by the application avoids the problems of repeated data insertion and missing log omission supplement, realizes efficient consistent replication of the database, and guarantees the stability and reliability of the database replication in a multi-cluster scenario.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a database replication method and system. Background Technology

[0002] Database replication, as a core technology for copying data operations from a source database to a target database, encompasses various forms such as periodic snapshots and continuous data streaming replication. Its core value lies in protecting database clusters from overall failures—when a cluster fails, a failover cluster maintaining data copies can restore data and allow operations to continue. However, with the increasing scale of database implementations, especially the growing prevalence of multi-cluster deployments in geographically distributed databases, ensuring efficient database replication remains a challenge. Data operations must be synchronized promptly and resource redundancy must be avoided to ensure that any copy remains up-to-date and available in the event of cluster failure. Furthermore, as a core means of improving database system throughput and fault tolerance, database replication faces increasingly prominent technical challenges in large-scale, distributed deployments, necessitating targeted optimization solutions to overcome application bottlenecks.

[0003] Under the current technological background, traditional high-performance replication solutions face two major drawbacks: topology incompatibility and chaotic sequence management, which severely restrict the application effect and system stability of database replication. Topology incompatibility manifests as difficulties in interoperability between replication protocols of different database versions, architectures, or vendors. This not only complicates the construction of cross-platform replication links but also increases system maintenance costs, thereby limiting architectural flexibility. Chaotic sequence management stems from imperfect generation and synchronization mechanisms for global identifiers such as auto-incrementing sequences and transaction IDs in a distributed environment. This easily leads to problems such as primary key conflicts, duplicate data insertion, or disordered transaction order, directly affecting the stability and data consistency of the replication link and reducing the overall fault tolerance of the system. Summary of the Invention

[0004] This invention aims to provide a database replication method and system to solve the above-mentioned technical problems, avoid the problems of duplicate data insertion and missing log transmission, achieve efficient and consistent database replication, and ensure the stability and reliability of database replication in multi-cluster scenarios.

[0005] To address the aforementioned technical problems, this invention provides a database replication method applied to a consumer cluster, comprising:

[0006] In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier, and then the database partition preset sequence number is obtained based on the associated partition identifier;

[0007] If the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then an independent numerical range is created based on the initial log source sequence number, and the target missing sequence number interval is identified based on the independent numerical range and the database partition preset sequence number.

[0008] A log retransmission signal is generated based on the target missing sequence number range and sent to the producer cluster, so that the producer cluster can feed back the missing sequence log based on the log retransmission signal;

[0009] Based on the missing sequence log, the independent value range and the preset sequence number of the database partition are filled in, the first sequence number of the database partition is obtained, and the database replication is completed.

[0010] The binary change log and the missing sequence log are obtained from the producer cluster.

[0011] In the above scheme, the database partition of the consumer cluster is located by associating partition identifiers and obtaining the preset sequence number of the database partition, laying the data foundation for subsequent sequence number continuity judgment. Next, under the premise that the initial log source sequence number and the preset sequence number of the database partition are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, an independent numerical range is created and the target missing sequence number interval is identified. This enables precise location and recording of missing data, avoiding incomplete data synchronization due to missing logs, and providing a clear target range for missing sequence log retransmission. Then, a log retransmission signal is generated based on the target missing sequence number interval and sent to the producer cluster, achieving precise location and targeted request for missing sequence logs. This ensures that the producer cluster can identify the range of missing sequence logs that need to be fed back based on the log retransmission signal, avoiding blind transmission and resource waste, and providing an accurate source of log data for filling data gaps. Finally, by filling the gap between the independent numerical range and the preset sequence number of the database partition through the feedback of missing sequence logs, the sequence number range is merged and made continuous. The latest applied sequence number of the database partition is obtained, that is, the first sequence number of the database partition. This enables data to be copied from the producer cluster to the consumer cluster, ensuring that the corresponding partition data of the consumer cluster and the producer cluster are consistent. This avoids the problems of duplicate data insertion and missing log transmission, and achieves efficient and consistent database replication, ensuring the stability and reliability of database replication in multi-cluster scenarios.

[0012] Furthermore, in response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier. Then, based on the associated partition identifier, a preset sequence number for the database partition is obtained, including:

[0013] In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier;

[0014] Based on the associated partition identifier, the database partition of the consumer cluster is determined;

[0015] Based on the consumer cluster, the database partition is partitioned, and the preset sequence number of the database partition is obtained.

[0016] In the above scheme, the binary change log is parsed in response to extract the initial log source sequence number, timestamp, and associated partition identifier. This collects the core metadata required in the replication process, providing data support for subsequent steps. Next, the extracted associated partition identifier is used to determine the database partition in the consumer cluster corresponding to the binary change log, achieving precise matching and ensuring that data changes are routed to the correct partition. This adapts to the partition mapping requirements in cross-topology deployment scenarios. Finally, by obtaining the preset sequence number of this database partition, a benchmark reference is provided for subsequent sequence number continuity verification and identification of target missing sequence number ranges.

[0017] Further, the step of establishing an independent numerical range based on the initial log source sequence number and the database partition preset sequence number if the initial log source sequence number and the timestamp are not consecutive, the initial log source sequence number is valid, and the initial log source sequence number meets the preset first condition, and then identifying the target missing sequence number interval based on the independent numerical range and the database partition preset sequence number, includes:

[0018] If the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then a new independent numerical range is created based on the initial log source sequence number;

[0019] Identify suspected missing sequence number ranges based on independent numerical ranges and database partition preset sequence numbers;

[0020] The suspected missing serial number range is verified version by version. If the version verification fails, the target missing serial number range is determined based on the timestamp of the suspected missing serial number range and the checksum of the suspected missing serial number range.

[0021] In the above scheme, under the premise that the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, an independent numerical range is created based on the initial log source sequence number. This provides a precise target range basis for the subsequent generation of log retransmission signals. Next, by combining the independent numerical range and the database partition preset sequence number to identify suspected missing sequence number intervals, the log transmission gaps can be initially located, avoiding the omission of potentially missing logs. Then, by performing a sequence-by-sequence version verification on the suspected missing sequence number intervals, if any version verification fails, the suspected missing sequence number intervals are filtered and corrected by combining the timestamp and checksum of the suspected missing sequence number interval, eliminating interference from invalid conflict logs, ensuring that an accurate target missing sequence number interval is obtained, and enabling the recording of missing logs in the binary change log.

[0022] Furthermore, the step of filling in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log to obtain the first sequence number of the database partition and complete the database replication includes:

[0023] Fill in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log to obtain the first sequence number of the database partition;

[0024] Record the first sequence number of the database partition, and perform DML operations on the binary change log and missing sequence log to complete the database replication.

[0025] In the above scheme, the gaps between independent numerical ranges and the preset sequence numbers of database partitions are filled by missing sequence logs, achieving sequence number merging and continuity. This yields the first sequence number of the database partition, restoring the integrity of the database partition sequence numbers in the consumer cluster and providing an accurate sequence number baseline. Next, by recording the first sequence number of the database partition and performing DML operations on the binary change log and missing sequence logs, data synchronization between the corresponding partitions of the consumer cluster and the producer cluster is achieved. Ultimately, efficient and consistent database replication is achieved, ensuring the stability and reliability of database replication in multi-cluster scenarios.

[0026] Furthermore, the step of filling in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log to obtain the first sequence number of the database partition includes:

[0027] The missing sequence logs are parsed to extract the source sequence number of the missing logs to be detected;

[0028] Perform conflict detection on the missing log source sequence number to be detected, and obtain the target missing log source sequence number;

[0029] Based on the target missing log source sequence number, fill in the independent numerical range and the database partition preset sequence number to obtain the first sequence number of the database partition.

[0030] In the above scheme, the missing sequence logs reported by the producer cluster are parsed to extract the source sequence numbers of the missing logs to be detected. This provides data support for subsequent conflict detection and sequence number filling, ensuring that subsequent operations can be accurately associated with the corresponding log data. Next, conflict detection is performed on the source sequence numbers of the missing logs to be detected, and target missing log source sequence numbers that meet the requirements are selected. This filters and corrects conflicting data in the missing sequence logs, preventing data inconsistencies between the consumer and producer clusters. Then, the gaps between the independent value ranges and the preset sequence numbers of the database partitions are filled using the target missing log source sequence numbers. This completes the merging and continuation of the independent value ranges and the preset sequence numbers of the database partitions, obtaining the first sequence number of the database partition. This restores the integrity of the database partition sequence numbers of the consumer cluster, providing an accurate and reliable sequence number benchmark for subsequent replication processes, and ensuring the stability and consistency of data replication in multi-cluster scenarios.

[0031] Furthermore, the step of filling in the independent numerical range and the preset sequence number of the database partition based on the target missing log source sequence number to obtain the first sequence number of the database partition includes:

[0032] Based on the target missing log source sequence number and the interval tree, the independent numerical range and the database partition preset sequence number are filled to obtain the first sequence number of the database partition.

[0033] In the above scheme, by combining the target missing log source sequence number with the interval tree algorithm, the gap between the independent numerical range and the preset sequence number of the database partition is accurately filled. This can merge and make the independent numerical range and the preset sequence number of the database partition continuous, thereby obtaining the first sequence number of the database partition. This ensures that the obtained first sequence number of the database partition can completely map the entire data synchronization state of the database partition, supporting efficient and stable data replication in cross-topology and multi-cluster scenarios.

[0034] Furthermore, it also includes:

[0035] If the initial log source sequence number and the database partition preset sequence number are consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then the first sequence number of the database partition is obtained based on the initial log source sequence number and the database partition preset sequence number, and the database replication is completed.

[0036] In the above scheme, under the premise that the initial log source sequence number and the preset sequence number of the database partition are consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, the preset sequence number of the database partition is updated with the initial log source sequence number to obtain the first sequence number of the database partition. This can achieve efficient data synchronization of the database partition of the consumer cluster, avoid redundant log retransmission process, improve the processing efficiency of the replication link, and finally complete efficient and reliable database replication, ensuring the consistency and real-time performance of data replication in multi-cluster scenarios.

[0037] Furthermore, it also includes:

[0038] If the timestamp is invalid or the initial log source sequence number does not meet the preset first condition, then database replication will stop, and the database partition preset sequence number will be maintained.

[0039] In the above scheme, under the premise that the timestamp is invalid or the initial log source sequence number does not meet the preset first condition, stopping the replication database can quickly intercept invalid and abnormal logs, preventing illegal or inconsistent data changes from being written to the consumer cluster database partition, thus ensuring the security and accuracy of data replication from the source. At the same time, maintaining the preset sequence number of the database partition when replication is stopped can stably retain the current data synchronization status of the consumer cluster database partition, preventing invalid logs from causing disorder in the database partition sequence number record, and providing a reliable baseline state for subsequently receiving valid change logs from the producer cluster and resuming the replication process, ensuring the fault tolerance and recoverability of the replication link.

[0040] This invention provides a database replication system applied to a consumer cluster, including a partition sequence number acquisition module, a missing interval identification module, a retransmission request sending module, and a missing log filling module, specifically:

[0041] The partition sequence number acquisition module is used to respond to the binary change log, parse the binary change log, extract the initial log source sequence number, timestamp and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier;

[0042] The missing interval identification module is used to create an independent numerical range based on the initial log source sequence number if the initial log source sequence number and the database partition preset sequence number are not continuous, the timestamp is valid, and the initial log source sequence number meets the preset first condition, and then identify the target missing sequence number interval based on the independent numerical range and the database partition preset sequence number.

[0043] The retransmission request sending module is used to generate a log retransmission signal based on the target missing sequence number range and send it to the producer cluster, so that the producer cluster can feed back the missing sequence log based on the log retransmission signal.

[0044] The missing log filling module is used to fill in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log, obtain the first sequence number of the database partition, and complete the database replication.

[0045] The binary change log and the missing sequence log are obtained from the producer cluster.

[0046] This invention provides a database replication system that, in practical applications, only requires a partition sequence number acquisition module. This module locates the database partition of the consumer cluster by associating with partition identifiers and obtains the preset sequence number of the database partition, laying the data foundation for subsequent sequence number continuity judgment. Next, a missing interval identification module is used. Under the premise that the initial log source sequence number and the preset database partition sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets a preset first condition, the system identifies the target missing sequence number interval and creates an independent numerical range. This allows for precise location and recording of missing data, avoiding incomplete data synchronization due to missing logs, and providing a clear target range for missing sequence log retransmission. Then, a retransmission request sending module generates a log retransmission signal based on the independent numerical range and the preset database partition sequence number and sends it to the producer cluster. This achieves precise location and targeted requests for missing sequence logs, ensuring that the producer cluster can identify the missing sequence log range to be fed back based on the log retransmission signal, avoiding blind transmission and resource waste, and providing an accurate source of log data for filling data gaps. Finally, a missing log filling module is used to fill the gap between the independent numerical range and the preset sequence number of the database partition by feeding back the missing sequence log. This completes the merging and continuation of the sequence number range, obtains the latest applied sequence number of the database partition, i.e., the first sequence number of the database partition, and realizes the replication from the producer cluster to the consumer cluster. This ensures that the corresponding partition data in the consumer cluster and the producer cluster are consistent, avoids the problems of duplicate data insertion and missing log transmission, achieves efficient and consistent database replication, and ensures the stability and reliability of database replication in multi-cluster scenarios.

[0047] Furthermore, the partition sequence number acquisition module is used to parse the binary change log in response to the binary change log, extract the initial log source sequence number, timestamp, and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier, including:

[0048] In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier;

[0049] Based on the associated partition identifier, the database partition of the consumer cluster is determined;

[0050] Based on the consumer cluster, the database partition is partitioned, and the preset sequence number of the database partition is obtained.

[0051] In the above scheme, the binary change log is parsed in response to extract the initial log source sequence number, timestamp, and associated partition identifier. This collects the core metadata required in the replication process, providing data support for subsequent steps. Next, the extracted associated partition identifier is used to determine the database partition in the consumer cluster corresponding to the binary change log, achieving precise matching and ensuring that data changes are routed to the correct partition. This adapts to the partition mapping requirements in cross-topology deployment scenarios. Finally, by obtaining the preset sequence number of this database partition, a benchmark reference is provided for subsequent sequence number continuity verification and identification of target missing sequence number ranges. Attached Figure Description

[0052] Figure 1 A flowchart illustrating a database replication method according to an embodiment of the present invention;

[0053] Figure 2 This is an architecture diagram of a database replication system provided in an embodiment of the present invention;

[0054] Figure 3 A one-way passive database system is provided in one embodiment of the present invention;

[0055] Figure 4 A bidirectional cross-datacenter database system is provided as an embodiment of the present invention;

[0056] in:

[0057] 101. One-way passive producer cluster; 102. One-way passive consumer cluster; 103. Two-way cross-datacenter producer cluster; 104. Two-way cross-datacenter consumer cluster; 111. One-way passive producer cluster client system; 112. One-way passive consumer cluster client system; 113. Two-way cross-datacenter producer cluster client system; 114. Two-way cross-datacenter consumer cluster client system; 120. Binary change log; 130. Snapshot. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] This embodiment provides a database replication method applied to a consumer cluster; please refer to its flowchart. Figure 1 ,include:

[0060] Step S1: In response to the binary change log, parse the binary change log, extract the initial log source sequence number, timestamp, and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier;

[0061] Step S2: If the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then create an independent numerical range based on the initial log source sequence number, and identify the target missing sequence number interval based on the independent numerical range and the database partition preset sequence number.

[0062] Step S3: Generate a log retransmission signal based on the target missing sequence number range and send it to the producer cluster, so that the producer cluster can feed back the missing sequence log based on the log retransmission signal;

[0063] Step S4: Fill in the independent value range and the preset sequence number of the database partition based on the missing sequence log, obtain the first sequence number of the database partition, and complete the database replication;

[0064] The binary change log and the missing sequence log are obtained from the producer cluster.

[0065] In this embodiment, the database clusters in the multi-cluster scenario are producer clusters and consumer clusters. The producer cluster can adopt a first topology, and the consumer cluster can adopt a second topology (different from the first topology). Since the consumer cluster has a different topology than the producer cluster, transactions corresponding to the binary change log may be routed to different consumer cluster database partitions, rather than to the consumer cluster database partition corresponding to the producer cluster database partition. This can be further understood as the replication flow of a single producer cluster database partition being split into multiple independent replication sub-flows on the consumer cluster, thereby executing independent database replication processes in different database partitions of the consumer cluster. In the producer cluster, when a write operation is performed on a database partition, the producer cluster's client system generates a binary change log containing the corresponding sequence number. This binary change log records the specific execution order of the write transaction, starting from the corresponding database partition sequence number 0 when the producer cluster is created. The partition allocation sequence number monotonically increases when writing binary change logs for that partition, thereby ensuring the reliability and consistency of the data replication process. This embodiment uses sequence number tracking technology. The sequence number can be combined with the identifier of the cluster or associated partition, for example, prefixed with the associated partition identifier. Next, the binary change log is transmitted from the producer cluster to the consumer cluster. The consumer cluster responds to the binary change log by parsing its header information, extracting the initial log source sequence number, timestamp, and associated partition identifier. Then, it uses the associated partition identifier to locate the database partition of the consumer cluster and obtain the preset sequence number of the database partition, laying the data foundation for subsequent sequence number continuity determination. Then, when the initial log source sequence number and the preset database partition sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets a preset first condition, an independent numerical range is created, and the target missing sequence number interval is identified. This allows for precise location and recording of missing data, avoiding incomplete data synchronization due to missing logs, and providing a clear target range for missing sequence log retransmission. The preset first condition specifically states that the initial log source sequence number monotonically increases when written to the binary change log, and the data in the initial log source sequence number corresponding to the binary change log is not duplicate, unordered, or other unsuitable data already stored by the consumer cluster. A deterministic hash function can be used to calculate the range to which the initial log source sequence number belongs, or a range partitioning strategy can be used to directly compare whether the initial log source sequence number and the preset database partition sequence number are consecutive. Then, a log retransmission signal is generated based on the target missing sequence number range and sent to the producer cluster. This enables precise location and targeted request for missing sequence logs, ensuring that the producer cluster can identify the range of missing sequence logs that need to be fed back based on the log retransmission signal. This avoids blind transmission and waste of resources, and provides an accurate source of log data for filling data gaps.Finally, by filling the gap between the independent numerical range and the preset sequence number of the database partition using the feedback missing sequence log, the sequence number range is merged and made continuous. The latest applied sequence number of the database partition is obtained, namely the first sequence number of the database partition. This enables data replication from the producer cluster to the consumer cluster. At the same time, based on the first sequence number of the database partition, it is ensured that the database partition of the consumer cluster has applied the binary change log, thereby ensuring that the corresponding partition data of the consumer cluster and the producer cluster are consistent. This avoids the problems of duplicate data insertion and missing log transmission, achieving efficient and consistent database replication and ensuring the stability and reliability of database replication in multi-cluster scenarios.

[0066] Since multiple non-contiguous ranges only occur when there are different topologies between the producer and consumer clusters, when the producer and consumer clusters use the same topology, the binary update log is configured to be applied sequentially from a given producer cluster database partition to a given consumer cluster database partition. For example, if a partition stores the applied ranges [1, 20] and [31, 50), then determining that sequence number 10 has been applied while sequence number 25 has not is fast and efficient. Furthermore, the sequence number ranges in this embodiment can be stored in numerical order for faster lookups. Another advantage of this approach is that when the sequence numbers in the applied binary change logs are consecutive, the required sequence number range is relatively streamlined. For example, if a consumer cluster database partition has successfully applied binary change logs for sequences 1 to 20, and subsequently successfully applied binary change logs for sequences 21 to 50, then that partition only needs to track the merged consecutive range [1, 50). Similarly, missing sequence logs, independent numerical ranges, and database partitions with default consecutive sequence numbers can be merged.

[0067] In this embodiment, write operations in the producer cluster can be performed by committing distributed transactions (MP transactions) on the producer cluster. Each database partition of the producer cluster generates a unit of work containing the associated partition identifier and its binary change log for the transaction. This unit of work is then inserted into the binary change log stream associated with the database partition of the producer cluster, and the unit of work is ordered relative to a single partition transaction executed on the same partition. For an MP transaction, each unit of work includes a sequence number from its original producer cluster's database partition. When the MP coordinator initiates an MP transaction on the consumer cluster, it aggregates the sequence numbers from all units of work, generates the corresponding binary change log, and then transmits it from the producer cluster to the consumer cluster.

[0068] Furthermore, in response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier. Then, based on the associated partition identifier, a preset sequence number for the database partition is obtained, including:

[0069] In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier;

[0070] Based on the associated partition identifier, the database partition of the consumer cluster is determined;

[0071] Based on the consumer cluster, the database partition is partitioned, and the preset sequence number of the database partition is obtained.

[0072] In this embodiment, the binary change log is parsed in response to extract the initial log source sequence number, timestamp, and associated partition identifier. This process collects the core metadata required for the replication process, providing data support for subsequent steps. Next, the extracted associated partition identifier is used to determine the database partition in the consumer cluster corresponding to the binary change log, achieving precise matching and ensuring that data changes are routed to the correct partition. This adapts to the partition mapping requirements in cross-topology deployment scenarios. Then, by obtaining the preset sequence number of this database partition, a benchmark reference is provided for subsequent sequence number continuity verification and identification of target missing sequence number ranges.

[0073] Further, the step of establishing an independent numerical range based on the initial log source sequence number and the database partition preset sequence number if the initial log source sequence number and the timestamp are not consecutive, the initial log source sequence number is valid, and the initial log source sequence number meets the preset first condition, and then identifying the target missing sequence number interval based on the independent numerical range and the database partition preset sequence number, includes:

[0074] If the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then a new independent numerical range is created based on the initial log source sequence number;

[0075] Identify suspected missing sequence number ranges based on independent numerical ranges and database partition preset sequence numbers;

[0076] The suspected missing serial number range is verified version by version. If the version verification fails, the target missing serial number range is determined based on the timestamp of the suspected missing serial number range and the checksum of the suspected missing serial number range.

[0077] In this embodiment, under the premise that the initial log source sequence number and the database partition preset sequence number are not continuous, the timestamp is valid, and the initial log source sequence number meets the preset first condition, an independent numerical range is created based on the initial log source sequence number, which can provide an accurate target range basis for the subsequent generation of log retransmission signals. Next, a three-stage filtering method is adopted to quickly identify obvious large-scale missing values ​​in the independent numerical range and the database partition preset sequence number through coarse-grained interval comparison. This can initially locate the log transmission gap, avoid missing potentially missing logs, and obtain the suspected missing sequence number interval. Then, the suspected missing sequence number interval is verified by serial number version by using an efficient bitmap compressed data structure (Roaring Bitmap). If there is a case where the version verification fails, that is, the sequence number has both version A and version B, the suspected missing sequence number interval is filtered and corrected by combining the microsecond-level timestamp and checksum of the suspected missing sequence number interval. The latest version is selected according to the timestamp and the data integrity is verified by the checksum to eliminate the interference of invalid conflict logs, ensuring that the accurate target missing sequence number interval is obtained, and the missing logs in the binary change log can be recorded.

[0078] Furthermore, the step of filling in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log to obtain the first sequence number of the database partition and complete the database replication includes:

[0079] Fill in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log to obtain the first sequence number of the database partition;

[0080] Record the first sequence number of the database partition, and perform DML operations on the binary change log and missing sequence log to complete the database replication.

[0081] In this embodiment, the gaps between independent numerical ranges and the preset sequence number of the database partition are filled by missing sequence logs, achieving sequence number merging and continuity, thereby obtaining the first sequence number of the database partition, restoring the integrity of the database partition sequence number of the consumer cluster, and providing an accurate sequence number benchmark. Next, by recording the first sequence number of the database partition and performing DML operations on the binary change log and missing sequence log in the Apply Binary Log transaction, the first sequence number of the database partition is recorded in the sequence number tracker corresponding to that database partition, realizing data synchronization between the corresponding partitions of the consumer cluster and the producer cluster, ultimately achieving efficient and consistent database replication, ensuring the stability and reliability of database replication in multi-cluster scenarios.

[0082] Furthermore, the step of filling in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log to obtain the first sequence number of the database partition includes:

[0083] The missing sequence logs are parsed to extract the source sequence number of the missing logs to be detected;

[0084] Perform conflict detection on the missing log source sequence number to be detected, and obtain the target missing log source sequence number;

[0085] Based on the target missing log source sequence number, fill in the independent numerical range and the database partition preset sequence number to obtain the first sequence number of the database partition.

[0086] In this embodiment, by parsing the missing sequence logs returned by the producer cluster, the sequence number of the missing log source to be detected is extracted, providing data support for subsequent conflict detection and sequence number filling, ensuring that subsequent operations can be accurately associated with the corresponding log data. Next, conflict detection is performed on the missing log source sequence number to be detected. Specifically, a primary key is defined for the missing log source sequence number to be detected using a basic Structured Query Language (SQL) table, and the primary key is used as the unique identifier for the tuple. When two tuples with the same primary key are inserted simultaneously in both the producer and consumer clusters, or when the same tuple is modified simultaneously in both the producer and consumer clusters (on a non-primary key column), a conflict is detected, and the tuple with the later timestamp is used in the consumer cluster. If a tuple is deleted in one cluster while a tuple in another cluster is modified simultaneously, a conflict is detected, and the tuple in both the producer and consumer clusters is deleted. These two solutions ensure that the clusters will eventually reach a consistent state after a conflict is detected. The process of selecting and obtaining the target missing log source sequence number that meets the requirements enables the filtering and correction of conflicting data in the missing sequence logs, preventing data inconsistencies between the consumer cluster and the producer cluster. Then, the gaps between the independent value range and the preset sequence number of the database partition are filled using the target missing log source sequence number. This completes the merging and continuation of the independent value range and the preset sequence number of the database partition, resulting in the first sequence number of the database partition. This restores the integrity of the consumer cluster's database partition sequence number, providing an accurate and reliable sequence number benchmark for subsequent replication processes, ensuring the stability and consistency of data replication in multi-cluster scenarios.

[0087] Furthermore, the step of filling in the independent numerical range and the preset sequence number of the database partition based on the target missing log source sequence number to obtain the first sequence number of the database partition includes:

[0088] Based on the target missing log source sequence number and the interval tree, the independent numerical range and the database partition preset sequence number are filled to obtain the first sequence number of the database partition.

[0089] In this embodiment, before filling, the PARTITION_SEQ_RANGE system table (field definitions: range_start, range_end, last_applied) maintained by each database partition can be directly scanned to collect non-contiguous intervals, i.e., gaps, between independent numerical ranges and the preset sequence number of the database partition. Then, by combining the target missing log source sequence number with the interval tree optimization merging algorithm, the gaps between the independent numerical ranges and the preset sequence number of the database partition are accurately filled. This merges and makes the independent numerical ranges and the preset sequence number of the database partition continuous, thereby obtaining the first sequence number of the database partition. This ensures that the obtained first sequence number of the database partition can completely map the entire data synchronization state of the database partition, supporting efficient and stable data replication in cross-topology and multi-cluster scenarios.

[0090] Furthermore, it also includes:

[0091] If the initial log source sequence number and the database partition preset sequence number are consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then the first sequence number of the database partition is obtained based on the initial log source sequence number and the database partition preset sequence number, and the database replication is completed.

[0092] In this embodiment, under the premise that the initial log source sequence number and the preset sequence number of the database partition are consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, the preset sequence number of the database partition is updated with the initial log source sequence number. This initial log source sequence number is then combined with the preset sequence number of the database partition in the consumer cluster, which has a start number and an end number, and stored in the consumer cluster to obtain the first sequence number of the database partition. The end number of the first sequence number of the database partition records the sequence number corresponding to the latest successfully applied binary log of the database partition. This enables efficient data synchronization of the database partitions in the consumer cluster, avoids redundant log retransmission processes, improves the processing efficiency of the replication link, and ultimately completes efficient and reliable database replication, ensuring the consistency and real-time performance of data replication in multi-cluster scenarios. Specifically, when the initial log source sequence number continuously follows the end number of the preset sequence number of the database partition, the end number of the preset sequence number of the database partition is replaced with the initial log source sequence number, thereby achieving seamless updating of the sequence number range. Taking account transfer as an example: The producer cluster (primary database) executes the transfer transaction, generating a binary update log with sequence number 205; the consumer cluster (standby database) receives this binary update log, detects that the database partition's preset sequence number is 204; verifies continuity (205 = 204 + 1), applies the transfer operation to the standby database account; updates the sequence number range to [1, 205], completes database replication, and achieves synchronization. The sequence number range tracked by each database partition can not only be used to identify the last applied binary change log of that database partition, but also completely map the full sequence of all successfully applied binary change logs since the database partition was created.

[0093] Furthermore, it also includes:

[0094] If the timestamp is invalid or the initial log source sequence number does not meet the preset first condition, then database replication will stop, and the database partition preset sequence number will be maintained.

[0095] In this embodiment, if the timestamp is invalid or the initial log source sequence number does not meet the preset first condition, stopping the database replication can quickly intercept invalid and abnormal logs, preventing illegal or inconsistent data changes from being written to the consumer cluster database partition, thus ensuring the security and accuracy of data replication from the source. At the same time, maintaining the preset sequence number of the database partition when replication is stopped can stably preserve the current data synchronization status of the consumer cluster database partition, preventing invalid logs from causing disorder in the database partition sequence number record, and providing a reliable baseline state for subsequently receiving valid change logs from the producer cluster and resuming the replication process, ensuring the fault tolerance and recoverability of the replication link.

[0096] This embodiment provides a database replication system; please refer to [link / reference]. Figure 2 It is applied to consumer clusters and includes a partition sequence number acquisition module, a missing interval identification module, a retransmission request sending module, and a missing log filling module, specifically:

[0097] The partition sequence number acquisition module is used to respond to the binary change log, parse the binary change log, extract the initial log source sequence number, timestamp and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier;

[0098] The missing interval identification module is used to create an independent numerical range based on the initial log source sequence number if the initial log source sequence number and the database partition preset sequence number are not continuous, the timestamp is valid, and the initial log source sequence number meets the preset first condition, and then identify the target missing sequence number interval based on the independent numerical range and the database partition preset sequence number.

[0099] The retransmission request sending module is used to generate a log retransmission signal based on the target missing sequence number range and send it to the producer cluster, so that the producer cluster can feed back the missing sequence log based on the log retransmission signal.

[0100] The missing log filling module is used to fill in the independent numerical range and the preset sequence number of the database partition based on the missing sequence log, obtain the first sequence number of the database partition, and complete the database replication.

[0101] The binary change log and the missing sequence log are obtained from the producer cluster.

[0102] This embodiment provides a database replication system that, in practical applications, only requires a partition sequence number acquisition module. This module locates the database partition of the consumer cluster by associating with partition identifiers and obtains the preset sequence number of the database partition, laying the data foundation for subsequent sequence number continuity judgment. Next, a missing interval identification module is used. Under the premise that the initial log source sequence number and the preset database partition sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets a preset first condition, it identifies the target missing sequence number interval and creates an independent numerical range. This enables precise location and recording of missing data, avoiding incomplete data synchronization due to missing logs, and providing a clear target range for missing sequence log retransmission. Then, a retransmission request sending module is used. This module generates a log retransmission signal based on the independent numerical range and the preset database partition sequence number and sends it to the producer cluster. This achieves precise location and targeted requests for missing sequence logs, ensuring that the producer cluster can identify the missing sequence log range that needs to be fed back based on the log retransmission signal, avoiding blind transmission and resource waste, and providing an accurate source of log data for filling data gaps. Finally, a missing log filling module is used to fill the gap between the independent numerical range and the preset sequence number of the database partition by feeding back the missing sequence log. This completes the merging and continuation of the sequence number range, obtains the latest applied sequence number of the database partition, i.e., the first sequence number of the database partition, and realizes the replication from the producer cluster to the consumer cluster. This ensures that the corresponding partition data in the consumer cluster and the producer cluster are consistent, avoids the problems of duplicate data insertion and missing log transmission, achieves efficient and consistent database replication, and ensures the stability and reliability of database replication in multi-cluster scenarios.

[0103] Furthermore, the partition sequence number acquisition module is used to parse the binary change log in response to the binary change log, extract the initial log source sequence number, timestamp, and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier, including:

[0104] In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier;

[0105] Based on the associated partition identifier, the database partition of the consumer cluster is determined;

[0106] Based on the consumer cluster, the database partition is partitioned, and the preset sequence number of the database partition is obtained.

[0107] In this embodiment, the binary change log is parsed in response to extract the initial log source sequence number, timestamp, and associated partition identifier. This process collects the core metadata required for the replication process, providing data support for subsequent steps. Next, the extracted associated partition identifier is used to determine the database partition in the consumer cluster corresponding to the binary change log, achieving precise matching and ensuring that data changes are routed to the correct partition. This adapts to the partition mapping requirements in cross-topology deployment scenarios. Then, by obtaining the preset sequence number of this database partition, a benchmark reference is provided for subsequent sequence number continuity verification and identification of target missing sequence number ranges.

[0108] To further illustrate the specific process of the initial log source sequence number arriving at the consumer cluster database partition and its relation to the database partition's preset sequence number, an example is provided:

[0109] 1. When the database partition preset sequence number list is empty and the initial log source sequence number is 1050, the sequence number 1050 is the first log to be applied. Since there is no database partition preset sequence number to follow continuously, the client system of the consumer cluster will create a new range [1050, 1050] to complete the database replication, indicating that the log with sequence number 1050 has been applied.

[0110] 2. When the preset sequence number range of the database partition is [1050, 1050] and the initial log source sequence number is 1051, the client system of the consumer cluster checks and finds that the sequence number 1051 is exactly the next number after the current maximum preset sequence number of the database partition, 1050. That is, the initial log source sequence number and the preset sequence number of the database partition are consecutive. At this time, the range is merged, and the ending number of the preset sequence number range of the database partition [1050, 1050] is replaced with the initial log source sequence number 1051, thereby expanding the preset sequence number range of the database partition and obtaining the first sequence number range of the database partition [1050, 1051]. This indicates that all logs from 1050 to 1051 have been applied consecutively.

[0111] 3. When the preset sequence number range of the database partition is [1050, 1051] and the initial log source sequence number is 1052, the client system of the consumer cluster checks and finds that the sequence number 1052 is exactly the next number after the current maximum preset sequence number of the database partition, 1051. That is, the initial log source sequence number and the preset sequence number of the database partition are consecutive. At this time, the range is merged, and the ending number of the preset sequence number range of the database partition [1050, 1051] is replaced with the initial log source sequence number 1052, thereby expanding the preset sequence number range of the database partition and obtaining the first sequence number range of the database partition [1050, 1052]. This indicates that all logs from 1050 to 1052 have been applied consecutively.

[0112] 4. When the database partition's preset sequence number range is [1050, 1052] and the initial log source sequence number is 1054, due to network latency, the log with sequence number 1053 has not yet arrived. The consumer cluster's client system checks and finds that sequence number 1054 does not continuously follow the current maximum database partition preset sequence number 1052, meaning the initial log source sequence number and the database partition preset sequence number are not continuous. In this case, range merging is not performed; instead, a new independent numerical range [1054, 1054] is created based on the initial log source sequence number, indicating that continuous logs from 1050 to 1052 have been applied, as well as the single log 1054. Simultaneously, the target missing sequence number interval [1053, 1053] can be identified based on the initial log source sequence number and the database partition preset sequence number. The consumer cluster's client system generates a log retransmission signal based on the target missing sequence number interval and sends it to the producer cluster, enabling the producer cluster to provide feedback on the missing sequence log based on the log retransmission signal.

[0113] 5. When the missing sequence log from the producer cluster reaches the consumer cluster, the default sequence number range for the database partition is [1050, 1052] and [1054, 1054]. The sequence number of the missing sequence log is 1053, which fills the gap between [1050, 1052] and [1054, 1054]. At this point, the client system of the consumer cluster performs range merging, connecting the default sequence number ranges [1050, 1052] and [1054, 1054] of the database partition to [1050, 1054] using the sequence number 1053 of the missing sequence log. This indicates that all logs from 1050 to 1054 have been applied consecutively, completing the database replication.

[0114] To further illustrate the specific process of the initial log source sequence number reaching the consumer cluster database partition and matching the preset sequence number of the database partition, an embodiment is provided: a geographically distributed database system is provided, including clusters 101-104 and their corresponding client systems 111-114. Disaster recovery can be divided into two modes: one-way passive and two-way cross-datacenter.

[0115] 1. One-way passive disaster recovery: Replicates data from the primary database to the replica database. See also... Figure 3In this mode, data replication is fixed in a one-way direction from the primary database to the replica database. Specifically, passive disaster recovery is achieved by replicating data from the one-way passive producer cluster 101 (primary database) to the one-way passive consumer cluster 102 (replica database). When an application on the one-way passive producer cluster client system 111 needs to perform routine operations such as INSERT (insert data), UPDATE (update / modify existing data), or DELETE (delete data), the one-way passive producer cluster client system 111 sends binary change logs 120 from the primary database to the replica database. These binary change logs 120 record the data modification process. If the relevant data already exists in the one-way passive consumer cluster 102 before the first database replication, the one-way passive producer cluster 101 can send a snapshot 130 of the existing data to the replica database. Once snapshot 130 is received and applied (at which point the one-way passive producer cluster 101 and the one-way passive consumer cluster 102 are synchronized), the one-way passive producer cluster client system 111 sends a binary change log 120 to record the transaction results to maintain synchronization between the clusters. To ensure consistency between the two databases on the one-way passive producer cluster 101 and the one-way passive consumer cluster 102, the one-way passive consumer cluster 102 can be initialized as a read-only database, allowing modification of its contents only based on transactions replicated from the one-way passive producer cluster 101.

[0116] 2. Bidirectional Cross-Data Center Disaster Recovery (XDCR): Supports bidirectional data synchronization. The bidirectional cross-data center producer cluster 103 and the bidirectional cross-data center consumer cluster 104 act as each other's producer and consumer clusters. Please refer to [link to relevant documentation]. Figure 4After a write operation is committed locally, the bidirectional cross-datacenter producer cluster 103 and the bidirectional cross-datacenter consumer cluster 104 generate binary update logs and push them to the other end, achieving bidirectional data synchronization. Applications on the bidirectional cross-datacenter producer cluster client system 113 and the bidirectional cross-datacenter consumer cluster client system 114 can perform read and write operations on their respective clusters. Changes in one database are synchronously replicated and applied to the other. This embodiment also provides conflict reporting and automatic solutions for cross-datacenter replication (XDCR), suitable for application scenarios where both clusters are writable clusters. When using XDCR, data conflicts may occur, where identical data, such as tuples, are modified in both clusters at the same or similar time points. When the clusters subsequently receive each other's data change logs, data conflicts will arise due to the aforementioned concurrent modification operations. To address such conflicts, this embodiment can automatically write conflict details to the exported table log, and the storage location of this log can be flexibly configured by the user, for example, it can be specified to be stored on different media such as local disk, Kafka message queue, or JDBC-compatible database.

[0117] This embodiment also provides data replication fault tolerance: if a specific node in the consumer cluster database fails (this specific node is a failover node), the failover node can retrieve the stored sequence number ranges for all database partitions to determine which binary change log sequence numbers have not yet been applied. Once determined, the failover node can request the missing sequence logs corresponding to the target missing sequence number range from the producer cluster, or generate new binary update logs as missing sequence logs starting from that point in time. Upon receiving the request, the producer cluster generates missing sequence logs corresponding to the target missing sequence number range and sends them to the failover node, enabling the failover node to receive the missing sequence logs, apply them to the corresponding database partitions, and update the sequence number ranges stored in the consumer cluster. The fault tolerance capability of MP transactions is similar to that of single partition transactions. The failover node collects data replication sequence number ranges from all partitions and requests the missing sequence logs corresponding to the target missing sequence number range from the producer cluster, or generates new binary update logs as missing sequence logs starting from that point in time. If the MP transaction is not executed by the consumer cluster, the corresponding MP work unit will be resent to the coordinator. This embodiment employs a fully ACID transaction processing mechanism and is implemented using a scalable relational database with a shared-nothing architecture, thus eliminating the risk of single point of failure.

[0118] In this embodiment, the producer cluster and consumer cluster also support parallel replication of multiple database partitions: multiple binary change logs associated with database partitions in the producer cluster can be received in parallel at the consumer cluster, and each binary change log carries a unique sequence number. Different binary change logs are applied to different database partitions in the consumer cluster. For each successfully applied binary change log, its sequence number is stored at the consumer cluster within a preset sequence number range associated with the database partition to which it was applied. This achieves parallel data processing, improving the speed and durability of the database system. Each partition can independently carry out the replication process, and multiple partition replication streams operate collaboratively without interference, ensuring the orderliness of the overall replication chain. Furthermore, the database replication in this embodiment is based on the binary change logs after transaction execution rather than the transactions themselves. Therefore, the consumer cluster does not need to reprocess transactions; it only needs to apply the change results to complete data synchronization, significantly reducing the computational overhead of the consumer cluster. Meanwhile, in a K-safe environment, if any server in the cluster fails, the replication streams of each partition can be flexibly redirected to other healthy nodes within the cluster to continue execution, or resumed after the failed server recovers, further enhancing the fault tolerance and reliability of multi-cluster data replication. Depending on the topology of the consumer cluster, the binary change logs generated by the producer cluster can be combined or not combined into a single ApplyBinaryLog transaction heading to the same consumer database partition. Finally, the sequence number status of each cluster database partition can be persistently stored on local storage media. When a recovery operation is performed on a database associated with that cluster database partition, these sequence numbers can be quickly retrieved and the database restored.

[0119] In this embodiment, the sequence number of the binary change log can also be used to detect whether the log is duplicated, so as to ensure that the ApplyBinaryLog transaction is idempotent: when the database partition of the consumer cluster receives the binary update log and is ready to execute the application operation, it will first query the current database partition preset sequence number range list of the database partition, and check whether the initial log source sequence number contained in the binary update log has been applied to the database partition. If the same sequence number is detected, the log is determined to be a duplicate log. Then the client system corresponding to the consumer cluster will discard or skip the application process of the log and will not execute the data change again, thereby achieving idempotency.

[0120] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A database replication method, characterized in that, Applied to consumer clusters, including: In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier, and then the database partition preset sequence number is obtained based on the associated partition identifier; If the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then a new independent numerical range is created based on the initial log source sequence number; Identify suspected missing sequence number ranges based on independent numerical ranges and database partition preset sequence numbers; The suspected missing serial number range is verified version by version. If the version verification fails, the target missing serial number range is determined based on the timestamp of the suspected missing serial number range and the checksum of the suspected missing serial number range. A log retransmission signal is generated based on the target missing sequence number range and sent to the producer cluster, so that the producer cluster can feed back the missing sequence log based on the log retransmission signal; The missing sequence logs are parsed to extract the source sequence number of the missing logs to be detected; Perform conflict detection on the missing log source sequence number to be detected, and obtain the target missing log source sequence number; Based on the target missing log source sequence number, fill in the independent numerical range and the database partition preset sequence number to obtain the first sequence number of the database partition; Record the first sequence number of the database partition, and perform DML operations on the binary change log and missing sequence log to complete the database replication; The binary change log and the missing sequence log are obtained from the producer cluster.

2. The database replication method according to claim 1, characterized in that, The response to the binary change log involves parsing the binary change log, extracting the initial log source sequence number, timestamp, and associated partition identifier, and then obtaining the database partition preset sequence number based on the associated partition identifier, including: In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier; Based on the associated partition identifier, the database partition of the consumer cluster is determined; Based on the consumer cluster, the database partition is partitioned, and the preset sequence number of the database partition is obtained.

3. The database replication method according to claim 1, characterized in that, The process of filling in the independent numerical range and the preset sequence number of the database partition based on the target missing log source sequence number to obtain the first sequence number of the database partition includes: Based on the target missing log source sequence number and the interval tree, the independent numerical range and the database partition preset sequence number are filled to obtain the first sequence number of the database partition.

4. The database replication method according to claim 1, characterized in that, Also includes: If the initial log source sequence number and the database partition preset sequence number are consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition, then the first sequence number of the database partition is obtained based on the initial log source sequence number and the database partition preset sequence number, and the database replication is completed.

5. A database replication method according to claim 1, characterized in that, Also includes: If the timestamp is invalid or the initial log source sequence number does not meet the preset first condition, then database replication will stop, and the database partition preset sequence number will be maintained.

6. A database replication system, characterized in that, Applied to consumer clusters, it includes a partition sequence number acquisition module, a missing interval identification module, a retransmission request sending module, and a missing log filling module, specifically: The partition sequence number acquisition module is used to respond to the binary change log, parse the binary change log, extract the initial log source sequence number, timestamp and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier; The missing interval identification module is used to create an independent numerical range based on the initial log source sequence number if the initial log source sequence number and the database partition preset sequence number are not consecutive, the timestamp is valid, and the initial log source sequence number meets the preset first condition; the missing interval identification module is used to identify suspected missing sequence number intervals based on the independent numerical range and the database partition preset sequence number. The missing sequence number identification module is used to perform serial number version verification on suspected missing sequence number intervals. If a version verification fails, the target missing sequence number interval is determined based on the timestamp of the suspected missing sequence number interval and the checksum of the suspected missing sequence number interval. The retransmission request sending module is used to generate a log retransmission signal based on the target missing sequence number range and send it to the producer cluster, so that the producer cluster can feed back the missing sequence log based on the log retransmission signal. The missing log filling module is used to parse the missing sequence log and extract the source sequence number of the missing log to be detected; The missing log filling module is used to perform conflict detection on the missing log source sequence number to be detected and obtain the target missing log source sequence number; The missing log filling module is used to fill the independent value range and the preset sequence number of the database partition based on the target missing log source sequence number to obtain the first sequence number of the database partition; the missing log filling module is used to record the first sequence number of the database partition and perform DML operations on the binary change log and the missing sequence log to complete the database replication; The binary change log and the missing sequence log are obtained from the producer cluster.

7. A database replication system according to claim 6, characterized in that, The partition sequence number acquisition module is used to respond to the binary change log, parse the binary change log, extract the initial log source sequence number, timestamp, and associated partition identifier, and then obtain the database partition preset sequence number based on the associated partition identifier, including: In response to the binary change log, the binary change log is parsed to extract the initial log source sequence number, timestamp, and associated partition identifier; Based on the associated partition identifier, the database partition of the consumer cluster is determined; Based on the consumer cluster, the database partition is partitioned, and the preset sequence number of the database partition is obtained.

Images