Data synchronization method, apparatus, system, device, medium, and product

Abstract

The present disclosure relates to the technical field of computers, and particularly provides a data synchronization method, device, system, equipment, medium and product. In the present disclosure, when a primary and backup data synchronization task is started, a first replication slot is determined from a configuration center, the first replication slot is used to record the progress of data synchronization, the configuration center is used to dynamically update and listen to slot information in the first replication slot; the slot information in the first replication slot is updated to the configuration center; when a new transaction is confirmed to be submitted, the current physical log is synchronized to a backup database of a first cluster; when a current log position of the backup database is received, the updated slot information is sent to the backup database. By modifying the transaction submission process, the decoupled synchronization of the slot information of the replication slot and the physical log is realized. Without modifying the corresponding logic of the physical log, the replication slot synchronization in the database one-primary-multiple-backup scenario is realized, and the efficiency of data synchronization is improved.

Description

Technical Field

[0001] This disclosure belongs to the field of computer technology, and specifically relates to a data synchronization method, apparatus, system, device, medium and product. Background Technology

[0002] The cloud-native database compute layer adopts a master-slave architecture, enabling master-slave failover in case of cluster failure. Logical replication slots are the core component of the database replication mechanism, primarily used to track and ensure reliable transmission and consistency maintenance of data changes. They capture data change events by parsing the database physical log (WAL), convert them into a logical format, and transmit them to the target. This is commonly used in database migration or active-active database setups. Due to the stateless nature of the compute layer, if the master database of the current cluster becomes unavailable due to a failure, master-slave failover typically occurs, with replication slots synchronizing to the slave databases. This is particularly important for data synchronization.

[0003] In related technologies, a write-ahead log can be added to display replication slot information, thereby modifying the physical log. The logical replication slot information between the primary and backup databases can be synchronized by sending the physical log. However, this data synchronization method involves frequent updates to the replication slots and introduces additional logs into the physical log, leading to excessive physical log size and wasted storage, which ultimately reduces data synchronization efficiency. Summary of the Invention

[0004] This disclosure addresses some of the shortcomings mentioned in the background art by providing a data synchronization method, apparatus, system, device, medium, and product that can improve the efficiency of data synchronization.

[0005] In a first aspect, embodiments of this disclosure provide a data synchronization method applied to the master database of a first cluster, comprising: When the primary and backup data synchronization task is started, the first replication slot is determined from the configuration center. The first replication slot is used to record the progress of data synchronization. The configuration center is used to dynamically update and monitor the slot information in the first replication slot. Update the slot information in the first copy slot to the configuration center; When a new transaction is confirmed to be committed, the current physical log is synchronized to the backup database of the first cluster; When the current log position of the backup database is received, the updated slot information is sent to the backup database.

[0006] Optionally, the method further includes: Call the lease creation interface to establish a lease relationship with the configuration center; Send first registration information to the configuration center. The first registration information includes at least one of the following: configuration information of the first cluster and the current log position.

[0007] Optionally, the method further includes: When a data migration task is started, the current log position is confirmed from the first replication slot. The data migration task is a task that migrates data from the first cluster to the second cluster. Synchronize the current physical logs to the primary database of the second cluster; Receive the current log position confirmed by the master database of the second cluster; Based on the current log position, update the slot information in the first replication slot to the configuration center.

[0008] Optionally, the method further includes: When a data migration task is initiated and the primary database of the first cluster fails, the data migration task is stopped.

[0009] Optionally, determining the first replication slot from the configuration center when starting the primary / standby data synchronization task includes: If the configuration center does not have the first replication slot, a request to create the first replication slot is sent to the configuration center.

[0010] In a second aspect, embodiments of this disclosure provide a data synchronization method applied to a backup database of a first cluster, comprising: When the primary-backup data synchronization task is started, the current physical logs from the primary database of the first cluster are received. Based on the current physical log, determine the current log position; When the current log position is a new position, the slot information of the first replication slot of the master database of the first cluster is obtained from the configuration center; the first replication slot is used to record the data synchronization progress of the master database of the first cluster, and the configuration center is used to dynamically update and monitor the slot information in the first replication slot; Based on the slot information of the first replication slot, the slot information of the second replication slot is updated; the second replication slot is used to record the progress of data synchronization of the backup database.

[0011] Optionally, the method further includes: When a data migration task is initiated and the primary database of the first cluster fails, the backup database is switched to execute the data migration task; the data migration task is a task of migrating data from the first cluster to the second cluster. Determine the current log position based on the new transaction received; The historical log position is determined from the second replication slot, and the historical log position is the last synchronized position information before the failure occurred; Based on the current log position and the historical log position, decode the corresponding physical log; The decoded physical log is sent to the master database of the second cluster.

[0012] Optionally, the method further includes: Call the lease creation interface to establish a lease relationship with the configuration center; Send second registration information to the configuration center. The second registration information includes a target listening event, which represents a change in the slot information of the first replication slot.

[0013] In a third aspect, embodiments of this disclosure provide a data synchronization method applied to a configuration center, comprising: Receive slot information from the master database of the first cluster. The slot information is the update information of the corresponding slot in the first replication slot, which is used to record the progress of data synchronization. Update the slot information in the first copy slot; Notify the backup database of the first cluster to update the slots; The slot information in the second replication slot is updated. The second replication slot is used to record the progress of data synchronization of the backup database.

[0014] Optionally, the method further includes: Receive the first registration information and / or the second registration information; Based on the creation request, the first replication slot and / or the second replication slot are created. The first replication slot includes at least one of the following slot information: current log position and historical log position.

[0015] In a fourth aspect, embodiments of this disclosure provide a data synchronization method applied to a second cluster, comprising: When the first cluster starts a data migration task, it receives physical logs from the first cluster; the data migration task is a task to migrate data from the first cluster to the second cluster. Based on the physical log, determine the current log position; Return the current log position to the first cluster.

[0016] In a fifth aspect, embodiments of this disclosure provide a data synchronization system, the system comprising: The primary database of the first cluster is used to execute the methods as described in any one of the first aspects; The backup database of the first cluster is used to execute the method as described in any one of the second aspects; Configuration center, used to execute the methods as described in any one of the third aspects; The second cluster is used to execute the methods described in the fourth aspect.

[0017] In a sixth aspect, embodiments of this disclosure provide a data synchronization apparatus applied to the master database of a first cluster, comprising: The first determination module is used to determine the first replication slot from the configuration center when the primary and backup data synchronization task is started. The first replication slot is used to record the progress of data synchronization. The configuration center is used to dynamically update and monitor the slot information in the first replication slot. The first update module is used to update the slot information in the first copy slot in the configuration center; The first synchronization module is used to synchronize the current physical log to the backup database of the first cluster when a new transaction is confirmed to be committed. The first sending module is used to send the updated slot information to the backup database when it receives the current log position of the backup database.

[0018] In a seventh aspect, embodiments of this disclosure provide a data synchronization apparatus applied to a backup database of a first cluster, comprising: The receiving module is used to receive the current physical log from the master database of the first cluster when the master-slave data synchronization task is started; The first determining module is used to determine the current log position based on the current physical log; The acquisition module is used to acquire the slot information of the first replication slot of the master database of the first cluster from the configuration center when the current log position is a new position; the first replication slot is used to record the data synchronization progress of the master database of the first cluster, and the configuration center is used to dynamically update and monitor the slot information in the first replication slot; The first update module is used to update the slot information of the second replication slot based on the slot information of the first replication slot; the second replication slot is used to record the progress of data synchronization of the backup database.

[0019] In an eighth aspect, embodiments of this disclosure provide a data synchronization device applied to a configuration center, comprising: The first receiving module is used to receive slot information from the master database of the first cluster. The slot information is the update information of the corresponding slot in the first replication slot, and the first replication slot is used to record the progress of data synchronization. The first update module is used to update the slot information in the first copy slot; The notification module is used to notify the backup database of the first cluster to update the slots; The second update module is used to update the slot information in the second replication slot, which is used to record the progress of data synchronization of the backup database.

[0020] In a ninth aspect, embodiments of this disclosure provide a data synchronization apparatus applied to a second cluster, comprising: The receiving module is used to receive physical logs from the first cluster when the first cluster starts a data migration task; the data migration task is a task of migrating data from the first cluster to the second cluster. The determination module is used to determine the current log position based on the physical log; The return module is used to return the current log position to the first cluster.

[0021] In a tenth aspect, embodiments of this disclosure provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described data synchronization method.

[0022] In an eleventh aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the above-described data synchronization method.

[0023] In a twelfth aspect, embodiments of this disclosure provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described data synchronization method.

[0024] In this disclosure, when the primary-standby data synchronization task is started, the first replication slot is determined from the configuration center. This first replication slot records the data synchronization progress, and the configuration center dynamically updates and monitors the slot information in the first replication slot. The slot information in the first replication slot is updated in the configuration center. When a new transaction is confirmed for commit, the current physical log is synchronized to the standby database of the first cluster. When the current log position is received from the standby database, the updated slot information is sent to the standby database. By modifying the transaction commit process, the currently generated physical log is synchronized to the standby database of the first cluster when a new transaction is confirmed for commit. Only after receiving the current log position reported by the standby database is the updated slot information accurately sent to the standby database, thus achieving decoupled synchronization between the replication slot's slot information and the physical log. This achieves replication slot synchronization in a primary-multiple-standby database scenario without modifying the corresponding physical log logic. Furthermore, it eliminates the need for frequent replication slot updates and the introduction of additional logs into the physical log, avoiding excessive physical log capacity and storage waste, thereby improving data synchronization efficiency.

[0025] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0026] Figure 1 This is a flowchart of a data synchronization method provided in this disclosure.

[0027] Figure 2 Another flowchart for a data synchronization method provided in this disclosure.

[0028] Figure 3 This is yet another flowchart of a data synchronization method provided in this disclosure.

[0029] Figure 4 Another flowchart for a data synchronization method provided in this disclosure.

[0030] Figure 5 This is an architecture diagram of a data synchronization method provided in this disclosure.

[0031] Figure 6 This is yet another flowchart of a data synchronization method provided in this disclosure.

[0032] Figure 7 This is yet another flowchart of a data synchronization method provided in this disclosure.

[0033] Figure 8 This is a schematic diagram of a data synchronization device provided in this disclosure.

[0034] Figure 9This is another schematic diagram of a data synchronization device provided in this disclosure.

[0035] Figure 10 This is another structural schematic diagram of a data synchronization device provided in this disclosure.

[0036] Figure 11 This is another schematic diagram of a data synchronization device provided in this disclosure.

[0037] Figure 12 This is a schematic diagram of the structure of a data synchronization system provided in this disclosure.

[0038] Figure 13 This is a hardware block diagram of an electronic device provided in this disclosure.

[0039] Figure 14 This is a schematic diagram of a computer program product provided in this disclosure. Detailed Implementation

[0040] To enable those skilled in the art to better understand the technical solution of this application, the application scenario of this application will be described first below.

[0041] HaiShan Database PostgreSQL is a cloud-native database with a storage-compute separation architecture. The compute layer accepts user requests independently and is stateless. The cloud-native database compute layer adopts a master-slave architecture, enabling master-slave failover in case of cluster failure. Logical replication slots are the core component of the database replication mechanism, primarily used to track and ensure reliable transmission and consistency maintenance of data changes. They capture data change events (such as inserts, updates, and deletes) by parsing the database physical log (WAL) and convert them into logical formats (such as JSON or Protobuf) before transmitting them to the target. This is commonly used for database migration or dual-active database setups. However, due to the stateless nature of the compute layer, logical replication slot failover has limitations. Replication slots can only be persisted locally on the master database. If the current master database becomes unavailable due to a failure, a master-slave failover typically occurs. Since the replication slot cannot be automatically synchronized to the slave database, the new master database cannot fully inherit the temporary state information of the previous master database, thus failing to provide the necessary synchronization task information for the replication slot, leading to interruption of the corresponding data synchronization tasks. Therefore, synchronizing the replication slot to the slave database is crucial for data synchronization.

[0042] In related technologies, a new plugin can be added to dynamically create and delete replication slots and query temporary information stored in replication slots. By modifying the current WAL (Write-Ahead Log) method—specifically by expanding the fields of the existing XLOG log type under the write-ahead log type—write-ahead logs containing replication slot information can be added. Logical replication slot information synchronization between the primary and standby databases is achieved through the sending of physical logs. Based on the primary database's replication slot information, a primary database physical log is generated. After being sent by the primary database's log sending process (walsender, WAL sender), the physical log is received by the standby database's log receiving process (walreceiver, WAL receiver). The standby database can then determine the replication slot information based on the physical log and write the information to disk, thus achieving replication slot synchronization between the primary and standby databases. However, this data synchronization method involves frequent updates to replication slots and introduces additional logs into the physical log, resulting in excessive physical log size and wasted storage, thereby reducing data synchronization efficiency. In addition, replication slots can be created at specified points within a past period. A custom replication slot (slot_reserved) can be created and maintained programmatically. At regular intervals (slotCheckTime), the replication slot is checked on the slave database. If it doesn't exist, the data structure of the custom replication slot (slot_reserved) is copied, and the slave database is restarted. This ensures high availability of the slot_reserved replication slot and guarantees that the function of creating replication slots at specified points is unaffected by database master-slave switching. However, this timed synchronization mechanism has limitations. If the time interval is too long, replication slot information may not be synchronized in time. If the time interval is too short, frequent synchronization can put pressure on the database itself. Furthermore, if a replication slot is found to be missing on the slave database, a database restart is required for the changes to take effect, which has a significant impact on business operations and reduces data synchronization efficiency.

[0043] To address the aforementioned technical issues, this disclosure provides an inventive concept: by modifying the transaction commit process, when confirming the commit of a new transaction, the currently generated physical log is synchronized to the standby database of the first cluster. After receiving the current log position reported by the standby database, the updated slot information is then accurately sent to the standby database, achieving decoupled synchronization between the slot information of the replication slot and the physical log. This achieves replication slot synchronization in a primary-multiple-standby database scenario without modifying the corresponding physical log logic. Furthermore, it eliminates the need for frequent updates to replication slots and avoids the introduction of additional logs into the physical log, preventing excessive physical log capacity and storage waste, thus improving data synchronization efficiency. Moreover, it ensures that the standby database of the first cluster can continue the previous replication slot tasks in the event of a failure of the primary database, achieving uninterrupted transmission tasks of the previous primary database and real-time breakpoint resumption during high-availability primary-standby failover.

[0044] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present disclosure and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the drawings, not the entire structure.

[0045] Figure 1 This is a flowchart illustrating a data synchronization method provided in this disclosure. Figure 1 As shown, this method is applied to the master database of the first cluster and includes: S101: When starting the primary / standby data synchronization task, determine the first replication slot from the configuration center.

[0046] Specifically, the first replication slot is used to record the progress of data synchronization. It is a persistent state component deployed locally on the primary database, used to accurately track the WAL consumption progress of the entire primary-standby synchronization link, mark the confirmed positions of downstream subscribers, and prevent the primary database from prematurely reclaiming WAL logs that have not yet been consumed by subscribers, ensuring that data changes are not lost or out of order. The configuration center is used to dynamically update and monitor the slot information in the first replication slot.

[0047] When maintenance personnel or automated scheduling platforms trigger the master-slave data synchronization process, the system does not directly create a temporary replication slot on the master database. Instead, it first sends a query request to the pre-connected configuration center. Based on the unique business identifier of the current synchronization task, it locates the complete definition information of the corresponding first replication slot, which is pre-registered or historically retained, from the configuration center. For example, in a synchronization scenario with breakpoint resumption, it directly reads the position information of the historical slot to avoid creating new slots repeatedly and causing progress loss. In the case of a brand new first synchronization task, it automatically creates a first replication slot that conforms to the specifications on the master database based on the preset environment parameters in the configuration center, and synchronizes and registers the basic metadata of the slot to the configuration center.

[0048] By determining the global definition of the first replication slot from the configuration center during the synchronization task startup phase, the global unified management and control of the replication slot information is realized. This completely avoids the problem in related technologies where the replication slot is only stored locally in the primary database and the new primary database cannot be aware of the synchronization task after the primary-standby switch. This significantly reduces the manual operation cost of the synchronization task initialization and ensures the recoverability of the synchronization task in the primary-standby switch scenario from the source.

[0049] S102: Update the slot information in the first copy slot to the configuration center.

[0050] Specifically, slot information refers to the full persistent state set of the first replication slot, used to completely record the running status and consumption progress of the replication slot. Whenever the state of the first replication slot on the primary database changes, the system automatically serializes the latest full slot information and pushes it to the corresponding configuration node in the configuration center in real time. This update process is not a full, timed overwrite, but rather uses an incremental change trigger mechanism. That is, whenever a downstream subscriber reports a new consumption confirmation point, the activation status of the replication slot on the primary database changes, or the slot's latency indicator fluctuates significantly, the system immediately captures this state change and pushes the updated slot information to the configuration center. Simultaneously, the configuration center automatically generates a historical version number for each slot update, preserving a complete change history and supporting subsequent version rollback and problem troubleshooting. Furthermore, the configuration center enables long-connection listening for this configuration node, and all backup database nodes in the cluster pre-subscribe to the change events of this configuration node. Once the slot information in the configuration center is updated, all subscribed backup nodes will immediately receive a push notification without needing to poll.

[0051] By updating all state changes of the first replication slot to the configuration center in real time, global near real-time synchronization and version management of slot information are achieved. This completely avoids the log bloat and version incompatibility issues caused by modifying physical logs in traditional solutions, and also eliminates the database performance pressure caused by periodic polling. This ensures the consistency and traceability of slot status in the global cluster.

[0052] S103: When confirming the commit of a new transaction, synchronize the current physical log to the backup database of the first cluster.

[0053] Specifically, the physical log is used to record all transaction changes in the database sequentially. After receiving a user's transaction commit request, completing the local physical log flushing operation, and confirming that the transaction has been persisted locally on the primary database, the primary database immediately sends the newly generated incremental physical log segment to all standby database nodes in the cluster via the native physical stream replication link. By synchronizing the current physical log to the standby database after a new transaction is committed and confirmed, reliable synchronization of underlying data changes between the primary and standby databases is achieved. This fully reuses the database's native capabilities without kernel modification, completely avoids the compatibility risks associated with custom physical logs, and thus ensures strong consistency of the underlying data in the primary and standby clusters, providing a complete data foundation for the subsequent storage of slot information in the standby database.

[0054] S104: When the current log position of the backup database is received, send the updated slot information to the backup database.

[0055] Specifically, the current log position refers to the log offset of the latest physical log that the standby database has successfully received and persisted locally. It is used to mark the current physical log synchronization progress of the standby database. After receiving the current log position confirmation message from the standby database, the primary database confirms that the physical logs in the standby database have been synchronized. Only then will the primary database send the latest complete slot information from the configuration center to the corresponding standby database. After receiving the slot information, the standby database writes it to its local replication slot persistence directory, directly completing the slot creation or update. The entire process does not require restarting the standby database process, and the new slot information takes effect immediately. This allows all standby databases to have a complete and usable first replication slot state without affecting business operations. After a primary-standby switchover, the new primary database can immediately continue the logical synchronization task without any link interruption issues.

[0056] This embodiment achieves uninterrupted synchronization of the database's native physical log. Compared to related technologies that add a new resource type as synchronization information for replication slots in the physical log and modify the database's physical log, this embodiment uses cloud-native middleware to update the dynamic information of replication slots in real time. The standby machine monitors the updates of the primary database's replication slots in real time and advances the current log position locally. This eliminates the need for synchronization of replication slot information through physical log transmission, reducing the volume of WAL (Write-Ahead Log) data and saving resources such as storage and network bandwidth.

[0057] In this disclosure, when the primary-standby data synchronization task is started, the first replication slot is determined from the configuration center. This first replication slot records the data synchronization progress, and the configuration center dynamically updates and monitors the slot information in the first replication slot. The slot information in the first replication slot is updated in the configuration center. When a new transaction is confirmed for commit, the current physical log is synchronized to the standby database of the first cluster. When the current log position is received from the standby database, the updated slot information is sent to the standby database. By modifying the transaction commit process, the currently generated physical log is synchronized to the standby database of the first cluster when a new transaction is confirmed for commit. Only after receiving the current log position reported by the standby database is the updated slot information accurately sent to the standby database, thus achieving decoupled synchronization between the replication slot's slot information and the physical log. This achieves replication slot synchronization in a primary-multiple-standby database scenario without modifying the corresponding physical log logic. Furthermore, it eliminates the need for frequent replication slot updates and the introduction of additional logs into the physical log, avoiding excessive physical log capacity and storage waste, thereby improving data synchronization efficiency.

[0058] In one possible implementation, the method is applied to the master database of a first cluster, and the method further includes: Call the lease creation interface to establish a lease relationship with the configuration center; send the first registration information to the configuration center.

[0059] Specifically, a lease relationship refers to a heartbeat binding contract with an expiration period established between the configuration center and the database computing node. This contract clarifies the current node's permissions and liveness status related to the configuration of a specified synchronization task. A periodic heartbeat renewal mechanism confirms the node's normal operation. If a node fails and becomes disconnected beyond the lease's expiration period, the configuration center automatically releases the relevant permissions held by the node, preventing the problem of zombie nodes consuming resources. The cloud-native database HaiShan PostgreSQL relies on the cloud and is directly orchestrated using Kubernetes. A cloud-native dynamic configuration center is configured within the cloud-native database cluster. After the primary database starts, it establishes a lease relationship with the cloud-native dynamic configuration center based on its local configuration. This is done by calling the LeaseGrant API (lease grant API) through a gRPC client. The kernel implements the Protocol Buffers protocol, registering current cluster information, including the cluster name and the current log position. The first registered information includes at least one of the following: the configuration information of the first cluster and the current log position.

[0060] By reporting cluster configuration information and the first registration information of the current log position to the configuration center, a global unified awareness of the status of all cluster nodes is achieved. This allows the configuration center to fully grasp the identity attributes and synchronization progress of all nodes, providing data support for the automatic selection of the optimal new master node during master-slave failover. This completely avoids the data loss problem caused by selecting a lagging node during master-slave failover, and greatly improves the reliability of cluster failover.

[0061] Subsequently, the database computing layer can directly utilize the cloud-native infrastructure distributed configuration center, with the primary database dynamically storing and updating the configuration center with replication slot information and other metadata.

[0062] In one possible implementation, the method is applied to the master database of a first cluster, and the method further includes: When the data migration task is started, the current log position is confirmed from the first replication slot; the current physical log is synchronized to the master database of the second cluster; the current log position is confirmed by the master database of the second cluster; and the slot information in the first replication slot is updated to the configuration center based on the current log position.

[0063] Specifically, a data migration task refers to the entire process of exporting existing data from the first source cluster, synchronizing changes through incremental synchronization, and finally smoothly switching business traffic to the second target cluster. This is used to achieve lossless migration of database data across clusters and versions. In other words, a data migration task is a task that migrates data from the first cluster to the second cluster.

[0064] During data migration, the system does not directly pull physical logs from arbitrary random points. Instead, it actively accesses the first replication slot pre-created on the first cluster master database, reads the latest position information recorded in that slot, and uses it as the starting reference point for the entire incremental migration phase. This avoids the problems of incremental starting point drift and premature physical log reclamation in traditional migration schemes, thus significantly improving the breakpoint resumption capability of cross-cluster data migration tasks and ensuring data integrity during the migration process. The first cluster master database continuously reads newly generated incremental physical logs from the starting point confirmed by the first replication slot, synchronizing these physical logs to the second cluster master database node in real time. Each time the second cluster master database completes the write-to-disk operation for a batch of incremental physical logs, it encapsulates the latest persisted log position into a confirmation message and sends it back to the first cluster master database via the reverse synchronization channel. Upon receiving this confirmation message, the first cluster master database does not directly modify the position of the first replication slot locally. Instead, it assembles the position into a slot status update request and sends it to the global configuration center. Upon receiving the request, the configuration center updates the `confirmed_flush_lsn` field in the slot information corresponding to the first replication slot to the latest position value and automatically generates a historical version record for this update, preserving a complete progress change trajectory. After the update is complete, the configuration center pushes the new slot status event to all nodes subscribed to that slot, and the first cluster master database then synchronously updates the local position value of the first replication slot, completing the final alignment between the local and global states.

[0065] By updating the slot information of the first replication slot to the configuration center based on the current log position confirmed by the second cluster, global persistence and multi-replica backup of the cross-cluster migration progress are achieved. This completely avoids the problem of loss of migration progress and task interruption caused by the failure of the primary database of the first cluster, thereby greatly improving the overall reliability of the cross-cluster data migration task and ensuring the stability of the task in large-scale cloud-native migration scenarios.

[0066] In one possible implementation, the method is applied to the master database of a first cluster, and the method further includes: If the data migration task is started and the primary database of the first cluster fails, the data migration task will be stopped.

[0067] Specifically, when the primary database in the cluster fails, if it is currently performing a data migration task, to ensure the data migration can continue, the primary database needs to be paused from performing the data migration task, and a primary-standby switchover needs to be completed so that the standby database can resume interrupted data transfer. Related technologies that periodically update replication slot information risk losing parts of the data that have already been advanced to the current log position, easily causing duplicate data migration and resulting in inconsistencies. This embodiment is based on the observer pattern, which monitors the status information of each logical replication slot task in the database cluster in real time, performs real-time advancement and updates locally, and in a one-primary-multiple-standby scenario, provides multiple replica redundancy, improving the reliability of the replication slots. When the cluster fails and a switchover to a new host occurs, interrupted data transfer can be resumed directly without affecting the business logic.

[0068] In one possible implementation, the method is applied to the primary database of the first cluster. An exemplary method for determining the first replication slot from the configuration center when a primary-standby data synchronization task is initiated includes: If the first replication slot does not exist in the configuration center, a request to create the first replication slot is sent to the configuration center.

[0069] Specifically, when a node starts a synchronization task and retrieves the first replication slot from the configuration center, if the configuration center returns a response indicating that the target slot does not exist, the primary database will not arbitrarily create the replication slot locally. Instead, it will immediately assemble a compliant first replication slot creation request, fully filling the request message with all relevant parameters of the current synchronization task, and send it to the configuration center. By sending a standardized creation request to the configuration center when the first replication slot does not exist, global unified management and uniqueness verification of the first replication slot are achieved. This completely avoids the name conflicts and permission chaos caused by arbitrarily creating slots locally in related technologies. Thus, from the initialization stage, the consistency of the first replication slot information across the entire cluster is guaranteed, providing a reliable foundation for seamless continuation of synchronization tasks in primary-standby switchover scenarios.

[0070] Figure 2 Another flowchart for a data synchronization method provided in this disclosure. Figure 2 As shown, this method is applied to the backup database of the first cluster, including: S201: When starting the master-slave data synchronization task, receive the current physical log from the master database of the first cluster.

[0071] Specifically, the standby database begins receiving current physical logs pushed by the primary database immediately after completing local instance startup and establishing a streaming replication connection with the primary database. This achieves efficient and lossless transmission of incremental data between the primary and standby databases, completely avoiding the performance bottlenecks and data consistency risks of logical synchronization, significantly reducing synchronization latency between the primary and standby databases, and ensuring real-time alignment between the standby and primary databases.

[0072] S202: Determine the current log position based on the current physical log.

[0073] Specifically, after the backup database completes the local persistence of each batch of received physical logs to disk, it calculates and determines the latest log position based on the end offset of this batch of logs. The backup database performs a continuity check between this position and the previously recorded position to confirm that the new position is strictly greater than the old position, ensuring that the positions are monotonically increasing and that there will be no abnormal position rollback. The backup database then returns the confirmation information of the current log position to the primary database.

[0074] S203: When the current log position is a new position, listen to the slot information of the first replication slot of the primary database of the first cluster in the configuration center.

[0075] Specifically, the first replication slot is used to record the data synchronization progress of the primary database in the first cluster, and the configuration center is used to dynamically update and monitor the slot information in the first replication slot. When the backup database detects that the current log position is a new increment position and the synchronization progress has made actual progress, it will not immediately execute subsequent operations, but will actively send a subscription request to the configuration center to start continuously listening for slot information change events in the first replication slot of the primary database in the first cluster. The backup database will register a dedicated event callback address in the configuration center. When the slot information of the first replication slot is updated, the configuration center will push the latest slot information to the backup database, without requiring the backup database to poll. When the backup database detects a new position, it starts listening to the primary database's first replication slot in the configuration center, achieving millisecond-level awareness of primary database slot information changes, completely avoiding the latency and resource waste caused by the polling mechanism, and allowing the backup database to grasp the global synchronization progress of the primary database in real time.

[0076] S204: Update the slot information of the second copy slot based on the slot information of the first copy slot.

[0077] Specifically, the second replication slot is used to record the data synchronization progress of the standby database. It is a logical replication slot dedicated to recording the data synchronization progress of the standby database itself. After receiving the latest slot information of the first replication slot pushed by the configuration center, the standby database performs a consistency check between the global synchronization point of the first replication slot of the primary database and its own current log point. After confirming that the difference between the two points is within a reasonable range, it updates the slot information of the second replication slot to a state that completely matches the actual synchronization progress of the standby database. The standby database will then synchronously report the updated slot information of the second replication slot to the configuration center, completing the final alignment of the global slot status.

[0078] The backup database updates its local second replication slot based on the global information of the first replication slot of the primary database, achieving global alignment of the slot progress at both ends of the primary and backup databases. This completely avoids the problem of the primary and backup slots being out of sync, and provides an accurate progress benchmark for the backup database to quickly take over as the new primary database when the primary database fails, greatly improving the reliability and speed of cluster primary-backup switching.

[0079] In one possible implementation, the method is applied to a backup database of a first cluster, and the method further includes: When a data migration task is initiated and the primary database of the first cluster fails, the backup database is switched to execute the data migration task; the current log position is determined based on the received new transactions; the historical log position is determined from the second replication slot, which is the last synchronized position information before the failure; the corresponding physical log is decoded based on the current log position and the historical log position; and the decoded physical log is sent to the primary database of the second cluster.

[0080] Specifically, the data migration task is the task of migrating data from the first cluster to the second cluster. A primary database failure refers to the state in which the primary node in the first cluster, responsible for data writing and physical log generation, loses its ability to provide services due to abnormal scenarios such as process crashes, disk damage, or network partitions. This failure triggers the cluster's primary / standby switchover process to prevent data inconsistency caused by the failed node continuing to provide services. Historical log positions refer to the last persistent position where the migration task had completed consumption and synchronized downstream before the failure of the old primary database in the first cluster. This is used to mark the progress boundary of the migration task before the failure occurred.

[0081] When a migration task is running and the primary database of the first cluster suddenly fails, the system will not directly interrupt the migration task. Instead, it will immediately trigger the primary / standby switchover process of the first cluster, promoting the pre-designated high-availability standby database to the new primary database. This new primary database will then fully take over the subsequent data migration tasks. After becoming the new primary database and completing its promotion and restoring external services, the standby database will continuously receive all new transactions submitted by the business side. After each transaction is committed and the changes are written to the physical log, the new primary database will determine the latest log position based on the offset position at the end of the physical log corresponding to that transaction. The new primary database will perform a validity check on this position to ensure that it is strictly greater than the last position before the old primary database failed, guaranteeing that the position is continuously increasing and that there will be no abnormal position jumps. Determining the current log position based on the new transactions received on the new primary database achieves accurate and true recording of the synchronization progress of the new primary database after the switchover, completely avoiding abnormal issues such as position jumps and inflated progress, and providing a completely reliable latest progress benchmark for the continued operation of subsequent migration tasks.

[0082] After the new primary database takes over the migration task, it does not directly start synchronization from the newly generated point. Instead, it actively accesses the pre-created second replication slot on the standby database and reads the last synchronized historical log point before the failure from the persistent metadata of that slot. The new primary database performs a validity check on this historical point, confirming that it has not been cleaned up by the log eviction mechanism within the current physical log range of the new primary database. It also confirms that this point is later than the snapshot point of the full migration phase, ensuring that there are no gaps between full and incremental data. By accurately reading the last synchronized historical log point before the failure from the second replication slot, the migration task failure point can be accurately traced back, completely avoiding the problem of complete loss of migration progress after the old primary database fails. This eliminates the need to re-execute the full migration from the beginning, significantly shortening the failure recovery time.

[0083] The new primary database uses historical log positions as the starting offset and the current log position as the ending offset to read all physical logs within this interval and decode them. The new primary database then pushes the decoded, continuously incremental physical logs to the primary database of the second cluster in real time, according to the log position order, to restore the transmission process. Sending the decoded, continuously incremental physical logs to the primary database of the second cluster in real time achieves efficient and lossless synchronization of incremental data within the faulty interval, quickly catching up on the migration delay caused by the primary database failure of the first cluster. This allows cross-cluster data migration tasks to quickly resume operation after a failure of the source primary database, without any data loss or inconsistency issues throughout the process.

[0084] In one possible implementation, the method is applied to a backup database of a first cluster, and the method further includes: Call the lease creation interface to establish a lease relationship with the configuration center; send the second registration information to the configuration center.

[0085] Specifically, the standby database establishes a lease relationship in the same way as the primary database. During initialization, it also establishes a lease relationship with the dynamic configuration center in advance and maintains a long-lived heartbeat connection. After establishing the long-lived connection, the standby database registers the dynamic events it wants to listen to with the configuration center. The standby database listens to the corresponding data structures in the configuration center, consumes the received listening information synchronously, and implements real-time updates of the replication slot metadata locally. This achieves real-time synchronization of the database replication slots. In the event of a cluster failure, it can continue the previous replication slot tasks, achieving breakpoint resumption and stability of the database data synchronization replication task. Specifically, it can listen for key-value changes under ` / cluster / slots` in the configuration center, including updates, deletions, and additions of current key-value pairs. Once the primary database makes a corresponding event change in the configuration center, it will actively notify each standby database through the listening service flow, and the standby database will execute the corresponding event to synchronize locally. The second registration information includes the target listening event, which represents a change in the slot information of the first replication slot.

[0086] Figure 3 This is yet another flowchart illustrating a data synchronization method provided in this disclosure. Figure 3 As shown, this method is applied to a configuration center and includes: S301: Receive slot information from the master database of the first cluster.

[0087] Specifically, the configuration center pre-opens a dedicated slot reporting access port. Once the primary database of the first cluster generates a positive update at its local first replication slot, it immediately encapsulates the latest slot information into a standard reporting message and pushes it to the configuration center via an encrypted channel. The slot information refers to the update information of the corresponding slot in the first replication slot, which is used to record the data synchronization progress. The configuration center securely receives the latest information from the primary database of the first cluster, achieving global synchronization of the primary database's local slot status. This completely breaks down the siloed barriers of slot information in traditional architectures, laying a reliable data foundation for unified management of slot status across the entire cluster.

[0088] S302: Update the slot information in the first copy slot.

[0089] Specifically, after the configuration center completes the legality verification of the reported slot information, it will not directly overwrite the original slot data. Instead, it will first generate a brand new version number for this update, write the new slot information into the configuration snapshot of this version, and then update the current pointer of the global configuration node of the first replicated slot to this new version.

[0090] S303: Notify the backup database of the first cluster to update the slots.

[0091] Specifically, after the configuration center completes the global information update of the first replication slot, it immediately iterates through the subscriber list for that slot and pushes the corresponding slot change event notification to all backup database nodes in the first cluster that have pre-registered and subscribed to the first replication slot. This achieves millisecond-level cluster-wide awareness of the first replication slot's state change, completely avoiding the latency and resource waste caused by the polling mode, and allowing all backup databases to obtain the latest synchronization progress of the master database immediately.

[0092] S304: Update the slot information in the second copy slot.

[0093] Specifically, the second replication slot is used to record the data synchronization progress of the standby database. After receiving the latest status of the local second replication slot from the standby database based on the update notification of the first replication slot, the configuration center verifies the legality of the status. It confirms that the position of the second replication slot of the standby database does not exceed the latest position of the first replication slot of the primary database, and there is no abnormal situation of the position being ahead. After the verification is passed, the information of the global configuration node of the second replication slot corresponding to the standby database is updated to the latest status, and the updated information is simultaneously synchronized and persisted to the multi-replica cluster of the configuration center.

[0094] In one possible implementation, the method is applied to a configuration center, and the method further includes: Receive first registration information and / or second registration information; based on the creation request, create first copy slot and / or second copy slot.

[0095] Specifically, the configuration center, as a cloud-native database that maximizes the reuse of cloud infrastructure, can dynamically update and monitor replication slot information, avoiding the introduction of other distributed storage components (such as distributed storage file systems for synchronous visibility of replication slots), thus simplifying the complexity of the database infrastructure. The cloud-native database compute layer directly utilizes the cloud-native infrastructure distributed configuration center, with the primary database dynamically storing and updating cluster replication slot information and other metadata to the configuration center. The first replication slot includes at least one of the following slot information: current log position and historical log position.

[0096] Figure 4 Another flowchart for a data synchronization method provided in this disclosure. Figure 4 As shown, this method is applied to the second cluster and includes: S401: When the first cluster starts the data migration task, it receives the physical logs from the first cluster.

[0097] Specifically, the data migration task involves migrating data from the first cluster to the second cluster. When the migration agent service on the second cluster officially starts the migration task from the first cluster to the second cluster, it will automatically open a dedicated physical log receiving port. The primary database of the first cluster will continuously push the generated physical logs to the migration agent of the second cluster in order of location through a pre-built streaming replication channel.

[0098] S402: Determine the current log position based on the physical log.

[0099] Specifically, after the migration agent persists the received physical logs to the physical log directory of the primary database of the second cluster, it calculates the latest log position based on the offset of the last valid log position of that physical log segment.

[0100] S403: Return the current log position to the first cluster.

[0101] Specifically, the migration agent of the second cluster encapsulates the confirmed current log position into a feedback message that conforms to the PostgreSQL native streaming replication protocol, and sends it back to the master database of the first cluster through the streaming replication channel between the second cluster and the first cluster.

[0102] Figure 5 This is an architecture diagram of a data synchronization method provided in this disclosure. Figure 5As shown, the primary and standby databases in the compute layer embed a gRPC client in the kernel. By implementing the corresponding logic within the kernel, they interact with the cloud-native infrastructure dynamic configuration center via gRPC when the database starts. The Raft consensus algorithm ensures the consistency of the overall cluster state. Both the primary and standby databases register their relevant information with the configuration center and establish corresponding lease information. As long as the primary and standby compute nodes do not fail, they will maintain a continuous connection and update their own information in the configuration center in real time, including the latest log position and temporary information about replication slots. Specifically, the cloud-native database HaiShan PostgreSQL relies on the cloud and is directly orchestrated using Kubernetes. A cloud-native dynamic configuration center is configured in the cloud-native database cluster. After the primary database starts, it establishes a lease relationship with the cloud-native dynamic configuration center based on its local configuration. This is done by calling the LeaseGrant API through the gRPC client and implementing the Protocol Buffers protocol within the kernel, registering current cluster information, including the cluster name and the latest log position. The remaining cloud-native standby databases follow the same approach as the primary database, establishing a lease relationship with the dynamic configuration center before initialization and maintaining a long-lived heartbeat connection. At the same time, after establishing a long-lived connection, it will register the dynamic events it wants to listen to, mainly the key-value changes under / cluster / slots in the configuration center, including the update, deletion, and addition of the current key-value pairs. Once the primary database makes the corresponding update in the configuration center, it will actively notify each backup node by relying on the listening service stream, and the backup node will execute the corresponding event to synchronize locally.

[0103] The detailed design data structure of the configuration center is shown below:

[0104] The following are descriptions of each key-value pair. The value corresponding to / cluster / slots / {slotname} is shown in the following structure information: { "slot_name": "test", "slot_type": "logical / physical", "confirmed_flush_lsn": "0 / 2D000028", "restart_lsn": "0 / 2D000028", "database": "db0", "datoid": "1635, "active": true} Figure 6 This is yet another flowchart illustrating a data synchronization method provided in this disclosure. Figure 6 As shown, the method includes: S601: The master database establishes a data synchronization task.

[0105] S602: Configuration Center updates or creates slot information.

[0106] Specifically, when the database cluster executes the start data synchronization task, the client first needs to create the corresponding replication slot by executing the corresponding command in the database. For example, the command `select pg_create_logical_replication_slot('test_slot', 'wal2json');` creates the corresponding replication slot in the primary database cluster, and the primary database cluster will first interact with the configuration center in the current command transaction.

[0107] S603: Determine if a new transaction has been committed.

[0108] Specifically, if a new transaction exists, S604 is executed; otherwise, the process ends.

[0109] S604: Logical parsing of physical logs in the main database.

[0110] Specifically, the primary database parses the physical logs in order to send them to the backup database.

[0111] S605: Synchronize physical logs from the primary database to the backup database.

[0112] Specifically, the standby database begins receiving current physical logs pushed by the primary database immediately after completing local instance startup and establishing a streaming replication connection with the primary database. This enables efficient and lossless transmission of incremental data between the primary and standby databases, completely avoiding the performance bottlenecks and data consistency risks of logical synchronization, significantly reducing synchronization latency between the primary and standby databases, and ensuring real-time alignment between the standby and primary databases.

[0113] S606: Determine if there are new log entries.

[0114] Specifically, if yes, then execute S607; otherwise, end.

[0115] S607: The configuration center monitors slot information changes and reports them to the backup database.

[0116] Specifically, the backup database registers a dedicated event callback address in the configuration center. When the slot information of the primary database is updated, the configuration center will push the latest slot information to the backup database, without requiring the backup database to poll and query.

[0117] S608: The backup database updates slots based on listener events.

[0118] Specifically, when the standby database detects a change in the corresponding key value, it operates in real time based on the corresponding event data stream. When the standby database detects a slot change, it first determines whether the current event is a slot addition or update event. If it's a slot addition event, it retrieves the slot information and creates the corresponding slot by executing `pg_create_logical_replication_slot("slotname")`. If it's a slot update event, it first checks if the standby database has the specified slot. If not, it creates the slot with the same name as the primary database. If the slot already exists locally, it retrieves the latest log position information of the slot in the event and uses its own function `pg_replication_slot_advance("confirmed_flush_lsn")` to move the replication slot to the latest position of that slot. If the detected event is a slot deletion event, it executes the `select pg_drop_replication_slot("test_slot")` function to delete the corresponding logical replication slot, thus ensuring complete consistency of the replication slot advancement log position information across the primary database and all standby databases.

[0119] Figure 7 This is yet another flowchart illustrating a data synchronization method provided in this disclosure. Figure 7 As shown, the method includes: S701: With the current transaction committed by the primary database, the log position is now advanced to log position 0 / 2D000028 on the primary database of the first cluster.

[0120] S702: Since the replication slot data synchronization task is configured at this time, the physical log will be decoded and transmitted to the second cluster for actual data playback and application.

[0121] S703: After receiving the parsed physical log, the second cluster completes the application locally and then sends a confirmation message to the primary database of the first cluster, indicating that it has received the log.

[0122] S704: After the primary database of the first cluster receives confirmation of the current log position from the second cluster, the current log position of the replication slot has been advanced to log position 0 / 2D000028 by default. At this time, the slot information in the dynamic configuration center is updated via gRPC, and the value of "confirmed_flush_lsn" corresponding to the key / cluster / slots / {slotname} is updated to the latest position information "0 / 2D000028".

[0123] S705: Since the backup databases of the first cluster have already registered with the configuration center and subscribed to slot change information, in S704, all backup databases of the first cluster will receive update events sent by the configuration center.

[0124] S706: All standby databases in the first cluster receive change events sent by the configuration center in S705. After parsing the change events, including event information such as (update / add / delete), taking an update as an example, since the position information of the replication slot is updated, after advancing, the position information of the replication slot is advanced locally.

[0125] S707: After the primary database of the first cluster fails and goes down, a primary-to-standby switchover occurs in the first cluster. Taking the new primary node of the first cluster as the standby database 1 as an example, the second cluster will re-establish a connection with the new primary database of the first cluster.

[0126] S708: After the new primary database of the first cluster receives the transaction commit request, the local current log position has advanced to position 0 / 2D000068. Since the position of the replication slot previously synchronized from the primary database is 0 / 2D000028, it is only necessary to decode the physical log data between 0 / 2D000028 and 0 / 2D000068. This ensures the continuity and accuracy of the data synchronization task of the second cluster.

[0127] S709: Send the decoded physical logs between decoded 0 / 2D000028 and 0 / 2D000068 to the second cluster for data replay and update.

[0128] S710: After the second cluster completes the playback of the corresponding physical log records, it sends a confirmation to the new master database of the first cluster that the physical logs have been played back, thereby completing the breakpoint reconnection of the data migration.

[0129] S711: After receiving the reply from the second cluster, the new primary database of the first cluster confirms the current log position record of the replication slot and updates it in the configuration center. At the same time, when other databases receive the update dynamics from the configuration center, they will also continuously update, thereby achieving synchronization of the replication slot.

[0130] When two database clusters perform data migration or replication tasks, if the primary database of the first cluster fails and a master-slave switch occurs, the latest standby machine has already synchronized the replication slot information on the primary node in real time and advanced to the current log position. Therefore, when the target replication slot establishes a database replication task with the new primary database of the first cluster, it can still continue the data replication task based on the current log position synchronized by the previous replication task, thereby achieving the ability to resume interrupted data transmission. This greatly saves data synchronization costs, improves the stability of data synchronization tasks, and lays the foundation for active-active databases.

[0131] In one possible implementation, in addition to synchronizing the replication slots after a cluster failure and high availability switchover, the method disclosed herein can also save temporary information on the primary database (including connection information, temporary transactions, etc.) so that the new primary database can completely simulate the state before the failure and be transparent and imperceptible to the business.

[0132] Figure 8 This is a schematic diagram of a data synchronization device provided in this disclosure. Figure 8 As shown, the device 800 is applied to the main database of the first cluster and includes: a first determination module 810, a first update module 820, a first synchronization module 830, and a first sending module 840.

[0133] The first determining module 810 is used to determine the first replication slot from the configuration center when the primary and backup data synchronization task is started. The first replication slot is used to record the progress of data synchronization. The configuration center is used to dynamically update and monitor the slot information in the first replication slot. The first update module 820 is used to update the slot information in the first copy slot to the configuration center; The first synchronization module 830 is used to synchronize the current physical log to the backup database of the first cluster when a new transaction is confirmed to be committed. The first sending module 840 is used to send the updated slot information to the backup database when it receives the current log position of the backup database.

[0134] Optionally, the device further includes: The calling module is used to call the lease creation interface and establish a lease relationship with the configuration center; The second sending module is used to send first registration information to the configuration center. The first registration information includes at least one of the following: configuration information of the first cluster and the current log position.

[0135] Optionally, the device further includes: The second determining module is used to determine the current log position from the first replication slot when a data migration task is started, wherein the data migration task is a task of migrating data from the first cluster to the second cluster. The second synchronization module is used to synchronize the current physical logs to the master database of the second cluster; The receiving module is used to receive the current log position confirmed by the master database of the second cluster; The second update module is used to update the slot information in the first replication slot to the configuration center based on the current log position.

[0136] Optionally, the device further includes: The stop module is used to stop the execution of the data migration task when the data migration task is started and the primary database of the first cluster fails.

[0137] Optionally, the first determining module is configured to: If the configuration center does not have the first replication slot, a request to create the first replication slot is sent to the configuration center.

[0138] Figure 9 This is another schematic diagram of a data synchronization device provided in this disclosure. Figure 9 As shown, the device 900 is applied to the backup database of the first cluster and includes: a receiving module 910, a first determining module 920, an obtaining module 930, and a first updating module 940.

[0139] The receiving module 910 is used to receive the current physical log from the master database of the first cluster when the master-slave data synchronization task is started; The first determining module 920 is used to determine the current log position based on the current physical log; The acquisition module 930 is used to acquire the slot information of the first replication slot of the master database of the first cluster from the configuration center when the current log position is a new position; the first replication slot is used to record the data synchronization progress of the master database of the first cluster, and the configuration center is used to dynamically update and monitor the slot information in the first replication slot. The first update module 940 is used to update the slot information of the second replication slot based on the slot information of the first replication slot; the second replication slot is used to record the progress of data synchronization of the backup database.

[0140] Optionally, the device further includes: A switching module is used to switch the backup database to perform the data migration task when a data migration task is started and the primary database of the first cluster fails; the data migration task is a task of migrating data from the first cluster to the second cluster. The second determination module is used to determine the current log position based on the new transaction received; The third determining module is used to determine the historical log position from the second replication slot, wherein the historical log position is the last synchronized position information before the failure occurred; The decoding module is used to decode the corresponding physical log based on the current log position and the historical log position; The first sending module is used to send the decoded physical log to the master database of the second cluster.

[0141] Optionally, the device further includes: The calling module is used to call the lease creation interface and establish a lease relationship with the configuration center; The second sending module is used to send second registration information to the configuration center. The second registration information includes a target listening event, which represents a change in the slot information of the first replication slot.

[0142] Figure 10 This is another structural schematic diagram of a data synchronization device provided in this disclosure. For example... Figure 10 As shown, the device 1000 is applied to a configuration center and includes: a first receiving module 1010, a first updating module 1020, a notification module 1030, and a second updating module 1040.

[0143] The first receiving module 1010 is used to receive slot information from the master database of the first cluster. The slot information is the update information of the corresponding slot in the first replication slot, and the first replication slot is used to record the progress of data synchronization. The first update module 1020 is used to update the slot information in the first copy slot; Notification module 1030 is used to notify the backup database of the first cluster to update the slots; The second update module 1040 is used to update the slot information in the second replication slot, which is used to record the progress of data synchronization of the backup database.

[0144] Optionally, the device further includes: The second receiving module is used to receive the first registration information and / or the second registration information; A creation module is used to create the first replication slot and / or the second replication slot based on a creation request. The first replication slot includes at least one of the following slot information: current log position and historical log position.

[0145] Figure 11 This is another schematic diagram of a data synchronization device provided in this disclosure. Figure 11 As shown, the device 1100 is applied to the second cluster and includes: a receiving module 1110, a determining module 1120, and a returning module 1130.

[0146] The receiving module 1110 is used to receive physical logs from the first cluster when the first cluster starts a data migration task; the data migration task is a task of migrating data from the first cluster to the second cluster. The determination module 1120 is used to determine the current log position based on the physical log; The return module 1130 is used to return the current log position to the first cluster.

[0147] Figure 12 This is a schematic diagram of the structure of a data synchronization system provided in this disclosure. Figure 12 As shown, the system 1200 includes: The primary database 1210 of the first cluster is used to execute methods that implement any one of the data synchronization methods applied to the primary database of the first cluster. The backup database 1220 of the first cluster is used to execute a method that implements any one of the data synchronization methods applied to the backup database of the first cluster; Configuration center 1230 is used to execute methods that implement any one of the data synchronization methods applied to the configuration center; The second cluster 1240 is used to execute methods that implement any one of the data synchronization methods applied to the second cluster.

[0148] This application also provides an electronic device for performing the above-described data synchronization method. Please refer to... Figure 13 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 13 As shown, the electronic device 130 includes: a processor 1300, a memory 1301, a bus 1302, and a communication interface 1303. The processor 1300, the communication interface 1303, and the memory 1301 are connected via the bus 1302. The memory 1301 stores a computer program that can run on the processor 1300. When the processor 1300 runs the computer program, it executes the data synchronization method provided in any of the foregoing embodiments of this application.

[0149] The memory 1301 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 1303 (which may be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0150] Bus 1302 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 1301 is used to store programs. After receiving an execution instruction, processor 1300 executes the program. The data synchronization method disclosed in any of the foregoing embodiments of this application can be applied to processor 1300, or implemented by processor 1300.

[0151] The processor 1300 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1300 or by instructions in software form. The processor 1300 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 1301. Processor 1300 reads the information in memory 1301 and completes the steps of the above method in conjunction with its hardware.

[0152] The electronic device provided in this application embodiment and the data synchronization method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0153] This application also provides a computer-readable storage medium corresponding to the data synchronization method provided in the foregoing embodiments. The computer-readable storage medium shown may be an optical disc, on which a computer program is stored. When the computer program is run by a processor, it executes the data synchronization method provided in any of the foregoing embodiments.

[0154] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0155] The computer-readable storage medium provided in the above embodiments of this application and the data synchronization method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0156] This application also provides a computer program product 1400, such as... Figure 14 As shown. This computer program product carries a computer program 1401. The instructions included in the program code can be used to execute the steps of the data synchronization method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0157] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0158] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0159] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0160] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0161] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0162] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0163] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0164] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A data synchronization method, characterized in that, The method, applied to the primary database of the first cluster, includes: When the primary and backup data synchronization task is started, the first replication slot is determined from the configuration center. The first replication slot is used to record the progress of data synchronization. The configuration center is used to dynamically update and monitor the slot information in the first replication slot. Update the slot information in the first copy slot to the configuration center; When a new transaction is confirmed to be committed, the current physical log is synchronized to the backup database of the first cluster; When the current log position of the backup database is received, the updated slot information is sent to the backup database.

2. The method according to claim 1, characterized in that, The method further includes: Call the lease creation interface to establish a lease relationship with the configuration center; Send first registration information to the configuration center. The first registration information includes at least one of the following: configuration information of the first cluster and the current log position.

3. The method according to claim 1, characterized in that, The method further includes: When a data migration task is started, the current log position is confirmed from the first replication slot. The data migration task is a task that migrates data from the first cluster to the second cluster. Synchronize the current physical logs to the primary database of the second cluster; Receive the current log position confirmed by the master database of the second cluster; Based on the current log position, update the slot information in the first replication slot to the configuration center.

4. The method according to claim 3, characterized in that, The method further includes: When a data migration task is initiated and the primary database of the first cluster fails, the data migration task is stopped.

5. The method according to claim 1, characterized in that, When the primary / standby data synchronization task is started, determining the first replication slot from the configuration center includes: If the configuration center does not have the first replication slot, a request to create the first replication slot is sent to the configuration center.

6. A data synchronization method, characterized in that, The method, applied to the backup database of the first cluster, includes: When the primary-backup data synchronization task is started, the current physical logs from the primary database of the first cluster are received. Based on the current physical log, determine the current log position; When the current log position is a new position, the slot information of the first replication slot of the master database of the first cluster is obtained from the configuration center; the first replication slot is used to record the data synchronization progress of the master database of the first cluster, and the configuration center is used to dynamically update and monitor the slot information in the first replication slot; Based on the slot information of the first replication slot, the slot information of the second replication slot is updated; the second replication slot is used to record the progress of data synchronization of the backup database.

7. The method according to claim 6, characterized in that, The method further includes: When a data migration task is initiated and the primary database of the first cluster fails, the backup database is switched to execute the data migration task; the data migration task is a task of migrating data from the first cluster to the second cluster. Determine the current log position based on the new transaction received; The historical log position is determined from the second replication slot, and the historical log position is the last synchronized position information before the failure occurred; Based on the current log position and the historical log position, decode the corresponding physical log; The decoded physical log is sent to the master database of the second cluster.

8. The method according to claim 6, characterized in that, The method further includes: Call the lease creation interface to establish a lease relationship with the configuration center; Send second registration information to the configuration center. The second registration information includes a target listening event, which represents a change in the slot information of the first replication slot.

9. A data synchronization method, characterized in that, Applied to a configuration center, the method includes: Receive slot information from the master database of the first cluster. The slot information is the update information of the corresponding slot in the first replication slot, which is used to record the progress of data synchronization. Update the slot information in the first copy slot; Notify the backup database of the first cluster to update the slots; The slot information in the second replication slot is updated. The second replication slot is used to record the progress of data synchronization of the backup database.

10. The method according to claim 9, characterized in that, The method further includes: Receive the first registration information and / or the second registration information; Based on the creation request, the first replication slot and / or the second replication slot are created. The first replication slot includes at least one of the following slot information: current log position and historical log position.

11. A data synchronization method, characterized in that, Applied to a second cluster, the method includes: When the first cluster starts a data migration task, it receives physical logs from the first cluster; the data migration task is a task to migrate data from the first cluster to the second cluster. Based on the physical log, determine the current log position; Return the current log position to the first cluster.

12. A data synchronization system, characterized in that, The system includes: The primary database of the first cluster is used to execute the method as described in any one of claims 1-5; The backup database of the first cluster is used to execute the method as described in any one of claims 6-8; Configuration center for implementing the method as described in any one of claims 9-10; The second cluster is used to execute the method as described in claim 11.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-5, 6-8, 9-10, and 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-5, 6-8, 9-10, and 11.

15. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the method as described in any one of claims 1-5, 6-8, 9-10, and 11.

Images