Transaction synchronization method, electronic equipment and storage medium

Abstract

The invention discloses a transaction synchronization method, electronic equipment and a storage medium, and the method comprises the steps: storing a transaction event written into a first system into a sending table, the sending table being a database table used for storing a to-be-synchronized event, and the transaction event being generated based on a local transaction in the first system; the transaction events meeting the preset state requirements in the sending table are pushed to a second system in batches; and in response to successful pushing, updating the state of the transaction event in the sending table. According to the scheme, the strong consistency of transaction data between heterogeneous systems can be improved.

Description

Technical Field

[0001] This application relates to the field of heterogeneous system synchronization technology, and in particular to a transaction synchronization method, electronic device, and storage medium. Background Technology

[0002] In practical applications, dual-write mode presents a significant challenge to ensuring data consistency when simultaneously writing transactional data to two heterogeneous systems: a relational database (Greenplum, GP) and a search engine (Elasticsearch, ES). In traditional dual-write solutions, the business system needs to write transactional data to both systems concurrently; failure to write to either system can lead to inconsistencies. Furthermore, the lack of a unified transaction coordination mechanism makes it difficult to guarantee the atomicity of cross-system operations, and the issue of transactional data consistency becomes particularly prominent in high-concurrency scenarios.

[0003] To address the challenge of transaction data consistency in dual-write mode, the industry urgently needs a technical solution that can both avoid the complexity of distributed transactions and guarantee strong consistency and near real-time synchronization of transaction data. Summary of the Invention

[0004] This application provides at least one transaction synchronization method, electronic device, and storage medium, which can improve the strong consistency of transaction data between heterogeneous systems.

[0005] The first aspect of this application provides a transaction synchronization method, which includes: saving transaction events written to a first system to a sending table, wherein the sending table is a database table used to store events to be synchronized, and the transaction events are generated based on local transactions in the first system; batch pushing transaction events in the sending table that meet preset state requirements to a second system; and updating the state of the transaction events in the sending table in response to successful push.

[0006] The preset state requirement is that the transaction event is in a state of pending synchronization. The transaction events in the sending table that meet the preset state requirement are pushed to the second system in batches. This includes: obtaining the number of events to be synchronized in the sending table, where the events to be synchronized are transaction events in the state of pending synchronization; and pushing the events to be synchronized to the second system in batches in response to the number of events meeting the preset quantity requirement.

[0007] The preset quantity requirement includes either the number of events being less than a first preset value or the number of events being greater than a second preset value. In response to the number of events meeting the preset quantity requirement, the events to be synchronized are pushed to the second system in batches, including: in response to the number of events being less than the first preset value, the events to be synchronized in the sending table are pushed to the second system in batches, with the first number of events to be synchronized as a batch; or, in response to the number of events being greater than the second preset value, the events to be synchronized in the sending table are pushed to the second system in batches, with the second number of events to be synchronized as a batch, wherein the second preset value is greater than the first preset value, and the second quantity is greater than the first quantity.

[0008] The process includes: after saving the transaction events written to the first system to the sending table, the following steps are taken: in response to the successful saving of the local transaction to the business table and the successful saving of the corresponding transaction event to the sending table, a transaction identifier for the local transaction is generated, and the corresponding transaction event stores the transaction identifier. The business table is a database table in the first system that stores transaction events. After the transaction events in the sending table that meet the preset status requirements are pushed to the second system in batches, the following steps are taken: in response to the transactions in the second system meeting the version update requirements, the corresponding transaction events in the second system are updated using the pushed transaction events, and the transaction identifier is used as the version number of the updated transaction event.

[0009] The transaction event information in the sending table includes the business table name written by the local transaction, the business table primary key value, the operation type, the data content, the transaction identifier, the status, and the creation time.

[0010] The method further includes: responding to push failure, waiting for the current interval duration, and then re-pushing the corresponding transaction event to the second system, wherein the current interval duration is the product of the preset interval duration and the number of re-pushing attempts; repeating the step of responding to push failure, waiting for the current interval duration, and then re-pushing the corresponding transaction event to the second system until the current interval duration is equal to or greater than the preset time threshold, stopping the push of the corresponding transaction event to the second system, and setting the status of the corresponding transaction event in the sending table to the failure status.

[0011] The method also includes: obtaining the creation time data of transaction events in the sending table that are in a state of pending synchronization; deleting transaction events whose creation time data is greater than or equal to a preset creation time threshold, and providing corresponding prompts.

[0012] Before pushing transaction events that meet the preset status requirements in the sending table to the second system in batches, the process includes locking the transaction events that meet the preset status requirements in the sending table.

[0013] A second aspect of this application provides an electronic device including a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the transaction synchronization method described in the first aspect above.

[0014] A third aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the transaction synchronization method described in the first aspect above.

[0015] The above scheme saves transaction events within the same local transaction written to the first system to a sending table. This sending table temporarily stores transaction events to be synchronized to the second system. Then, it selects transaction events from the sending table that meet preset status requirements and pushes them in batches to the second system. If the push is successful, the status of the transaction events in the sending table is updated. This design, through the "intermediate storage + status control" mechanism of the sending table, can accurately guarantee strong consistency of transaction data between the first and second systems (i.e., heterogeneous systems). Simultaneously, the batch push processing mode significantly reduces the frequency of interactions, effectively improving the synchronization efficiency of event data.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the transaction synchronization method of this application; Figure 2 This is a flowchart illustrating another embodiment of the transaction synchronization method of this application; Figure 3 This is a schematic diagram of the framework of an embodiment of the transaction synchronization device of this application; Figure 4 This is a schematic diagram of the framework of an embodiment of the electronic device of this application; Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0019] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0021] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the transaction synchronization method of this application. Specifically, it may include the following steps: Step S110: Save the transaction events written to the first system to the sending table, wherein the sending table is a database table used to store events to be synchronized, and the transaction events are generated based on local transactions in the first system.

[0023] This application primarily applies to the field of heterogeneous system technology. It stores events to be synchronized in a first system using a sending table, and then pushes these events in batches to a second system, thereby ensuring strong consistency of transaction data between the first and second systems. In the synchronization link of this application, there is no dual-write window; the write path only passes through the first system, thus reducing write latency.

[0024] The first and second systems can be heterogeneous systems such as relational databases (e.g., MySQL, PostgreSQL) + caches (e.g., Redis, Memcached), relational databases (e.g., MySQL, Oracle) + message queues (e.g., Kafka, RocketMQ, Canal), relational databases (e.g., MySQL, PostgreSQL, GP) + search engines (e.g., ES, Apache Solr), and caches (e.g., Redis) + message queues (e.g., Kafka, RocketMQ), without specific limitations.

[0025] In this embodiment, taking a relational database (GP) as the first system and a search engine (ES) as the second system as an example, the specific details involve a method and system for strong consistency and near real-time data synchronization between the relational data warehouse (GP) and the search engine (ES). The business system includes the relational data warehouse (GP) and the search engine (ES). The business system can directly complete the read and write operations of core business data through the Greenplum database (GP), without relying on other middleware or service layers (such as message queues, caches, etc.). Within the same local transaction, in addition to writing to the business table, a row of events (i.e., transaction events) is inserted into the Outbox table (i.e., the sending table), and gp_txid (i.e., transaction identifier) ​​is automatically populated. Here, a local transaction means that all operations (such as inserting into the order table and updating the inventory table) performed in a single database connection are wrapped within a transaction boundary, and either all are successfully committed or all are rolled back.

[0026] In addition, when a transaction event is written to the Outbox table, the Outbox table will also record relevant information about the transaction event, such as the business table name (table_name), primary key value (primary_key), operation type (op_type), data content, transaction identifier (gp_txid), status (status), and creation time (create_time) of the local transaction.

[0027] In some implementations, the sending table may be: CREATE TABLE es_outbox( id bigserial primary key, table_name text, -- Business table name primary_key text, -- primary key value of the business table op_type char(1), -- Operation type: 'I'=INSERT, 'U'=UPDATE, 'D'=DELETE payload jsonb, -- complete data content, used for synchronization to Elasticsearch. `gp_txid bigint default gp_txid(), -- GP transaction ID, globally incrementing, used as the Elasticsearch version number to ensure ordering` status smallint default 0, -- Synchronization status: 0 = pending, 1 = completed, -1 = failed `create_time timestamp default now()` -- Creation time `distributed by(id)`: -- GP distribution key, distributes data to nodes by ID.

[0028] In some implementations, after inserting a transaction event into the sending table within the same local transaction, in response to the successful saving of the local transaction to the business table and the successful saving of the corresponding transaction event to the sending table, i.e., after the local transaction is successfully committed, the first system will generate a transaction identifier corresponding to the local transaction, and the transaction event within the local transaction will also have a transaction identifier, establishing a strong association between the event and the transaction. The business table is a database table in the first system that stores transaction events.

[0029] Furthermore, after a local transaction is successfully committed, the synchronizer in the business system polls the sending table and locks transaction events in the sending table that meet preset status requirements, for example, through the SELECT ... FOR UPDATE SKIPLOCKED mechanism. This mechanism automatically skips events that have been locked by other sessions during the query, ensuring mutual exclusion of event retrieval. The synchronizer service can execute specific SQL statements such as SELECT * FROM outbox WHERE status = 0 FOR UPDATE SKIP LOCKED when polling the Outbox table. This database locking mechanism prevents multiple instances from processing the same event simultaneously. For example, when multiple synchronizer instances query concurrently, only the first instance acquires the lock and processes the event; subsequent instances automatically skip, avoiding data contention. This locking mechanism relies on Greenplum's distributed transaction features, requires no additional application-layer synchronization logic, and can efficiently support multi-instance parallel processing while maintaining event processing order and consistency.

[0030] By performing event locking before pushing, the system effectively avoids duplicate synchronization or state anomalies caused by multiple Syner instances processing the same event simultaneously, significantly improving the concurrent processing capability and reliability of data synchronization. This mechanism leverages Greenplum's native locking features to achieve lock-free event consumption, reducing the risk of synchronization failures caused by concurrent access, ensuring the orderliness and idempotency of cross-system operations, thereby supporting stable data synchronization in high-throughput scenarios, while providing a fundamental guarantee for system horizontal scaling and avoiding the problems of manual compensation or latency accumulation in traditional solutions.

[0031] If a local transaction commit fails, the transaction is rolled back and the transaction synchronization ends.

[0032] In one implementation, the transaction identifier refers to the distributed transaction identifier `gp_txid` automatically generated by the GP database system. This identifier is globally unique and monotonically increasing, used to uniquely identify the transaction commit order. `gp_txid` can be used as a field in the Outbox table, automatically populated when a business transaction commits. For example, when a business transaction writes to Greenplum's business table, an event row carrying `gp_txid` is synchronously inserted into the Outbox table, ensuring atomic commit between the business table and the Outbox table. When the Syner service polls the Outbox table, it obtains the events to be synchronized through the `SELECT ... FOR UPDATE SKIP LOCKED` mechanism, passes `gp_txid` as the version number to Elasticsearch, and sets the `version_type=external` and `version=gp_txid` parameters in the write operation to achieve idempotent overwrite updates of documents.

[0033] By using the transaction identifier as a version number for the second system update, its monotonically increasing characteristic ensures the idempotency of Elasticsearch document writes and avoids inconsistencies caused by duplicate data writes. At the same time, the transaction identifier is generated when the business transaction is committed and atomically saved with the business data, ensuring the sequentiality of cross-system operations, eliminating the need for compensation caused by data backlog or write failures, and significantly improving the reliability and real-time performance of distributed data synchronization.

[0034] In one specific embodiment, after a user submits an order, the business system writes the order data in a GP transaction and inserts a transaction event (such as order_created) to be sent into the Outbox table within the same local transaction, recording the event type, data, and associated GP transaction ID (gp_txid). After the GP transaction is committed, a scheduled task scans the Outbox table, filters the events of committed transactions, and sends them to a message queue (such as Kafka) so that consumers can subsequently pull events from the queue and call the ESAPI to update the index. If sending fails, eventual consistency is ensured through retries or a dead-letter queue.

[0035] Step S120: Batch push transaction events that meet the preset status requirements in the sending table to the second system.

[0036] In some implementations, the business system also includes a synchronizer, which uses the synchronizer service to poll the send table to obtain transaction events in the send table that meet preset state requirements.

[0037] Specifically, the preset state requirement is that the transaction event is in a pending synchronization state. The Syner is used to obtain the number of pending events in the sending table, where each pending event is a transaction event in the pending synchronization state. In response to the event count meeting the preset requirement, the pending events are pushed to the second system in batches. Syner can be set to query periodically, such as every 1 minute or 10 minutes; no specific limitation is made here.

[0038] In other implementations, the Binlog sending table can be monitored to obtain the number of events to be synchronized in the sending table. When the number of events meets the preset requirement, the events to be synchronized are pushed to the second system in batches.

[0039] Furthermore, the preset quantity requirement includes either the number of events being less than a first preset value or the number of events being greater than a second preset value. After obtaining the number of events, in response to the number of events being less than the first preset value, the events to be synchronized in the sending table are pushed to the second system in batches, with the first quantity of events as a batch. Alternatively, in response to the number of events being greater than the second preset value, the events to be synchronized in the sending table are pushed to the second system in batches, with the second quantity being greater than the first preset value and the second quantity being greater than the first quantity.

[0040] For example, when the number of events is less than 1000, 500 events are pushed in batches; when the number of events is greater than 10000, 2000 events are pushed in batches. The Synker service automatically triggers batch size switching by polling the Outbox table for event status in real time and calculating the current queue length. This mechanism utilizes the monotonically increasing characteristic of the first system's local transaction identifier (gp_txid) to ensure the push order, avoiding resource waste or processing delays caused by rigid batching strategies. It also works in conjunction with the exponential backoff retry mechanism to smoothly adjust the request frequency when the second system service experiences fluctuations, preventing retry surges from impacting system stability.

[0041] By dynamically adjusting batch size, the business system reduces resource consumption under short queues when the number of events is low, avoiding network overhead and write pressure on the second system caused by frequent small-batch pushes; when the number of events is high, it increases the processing speed of long queues, accelerates the data synchronization process, and effectively prevents jitter and latency caused by request overload in the second system service. This adaptive mechanism significantly optimizes resource utilization and system throughput, ensuring efficient and stable operation of data synchronization under different load scenarios, while maintaining strong consistency with transaction identifier version control.

[0042] In some implementations, after the transaction events that meet the preset status requirements in the sending table are pushed to the second system in batches, in response to the transactions in the second system meeting the version update requirements, the corresponding transaction events in the second system are updated using the pushed transaction events, and the transaction identifier is used as the version number of the updated transaction event.

[0043] For example, if the current version of the document corresponding to the event in the second system is 1.0, while the version of the synchronized event, i.e. the transaction identifier, is 2.0, then since 2.0 > 1.0, the current version of the document corresponding to the event in the second system will be updated to 2.0.

[0044] Step S130: In response to successful push, update the status of the transaction event in the sending table.

[0045] In some implementations, if the push is successful, the status of the transaction event in the sending table is set to status=1.

[0046] Furthermore, if the push fails, after waiting for the current interval, the corresponding transaction event will be pushed to the second system again. The current interval is the product of the preset interval and the number of times to push again. The process of responding to the push failure, waiting for the current interval, and then pushing the corresponding transaction event to the second system again will be repeated until the current interval is equal to or greater than the preset time threshold. At this point, the push of the corresponding transaction event to the second system will be stopped, and the status of the corresponding transaction event in the sending table will be set to the failure status.

[0047] This implementation employs an exponential backoff retry strategy. The initial preset interval can be set to 1 second, and each subsequent retry multiplies this initial value by 2. The preset time threshold can be set to 60 seconds. When a push fails, the first retry waits 1 second, the second 2 seconds, the third 4 seconds, and so on, with the retry interval doubling exponentially. When the retry interval reaches or exceeds 60 seconds, the system automatically stops retries and sets the event status in the sending table to status=-1, preventing infinite retry loops from consuming system resources. Through this mechanism, the business system can dynamically adjust the retry rhythm during ES service jitter or network fluctuations, effectively reducing the impact of short-term high-concurrency requests on downstream services.

[0048] This mechanism effectively avoids retry surges caused by frequent retries when Elasticsearch (ES) is unstable. Through exponentially increasing retry intervals, the system adaptively reduces the retry frequency, alleviating the load on ES and preventing further service degradation. When the retry interval reaches a preset time threshold, the system promptly terminates retries and marks the failure status, avoiding unnecessary resource consumption and providing a clear failure identifier for the automatic compensation loop. This design significantly improves the stability and robustness of data synchronization, ensuring rapid convergence of the data synchronization link under abnormal conditions, reducing the need for manual intervention, and guaranteeing the consistency between business data and ES indexes.

[0049] In some implementations, to reduce the problem of excessive system resource consumption caused by event backlog, an automatic cleanup mechanism can be set up in the business system. Specifically, the creation time data of transaction events in the sending table that are in a pending synchronization state can be obtained, transaction events whose creation time data is greater than or equal to a preset creation time threshold can be deleted, and corresponding prompts can be given.

[0050] Obtaining the creation time (create_time) data of events to be synchronized in the sending table refers to periodically scanning the sending table through a real-time detection module and extracting the create_time field value of each record. Deleting expired events is based on a preset creation time threshold (e.g., 24 hours). When an event's creation time exceeds this threshold, the corresponding event is removed and a system notification is triggered. This can be achieved using a scheduled task mechanism, such as executing a database query every 5 minutes to filter events with status=0 and create_time greater than the preset threshold, performing deletion operations, and generating an alarm log. Alternatively, a TTL (Time to Live) attribute can be attached when inserting events, and the system will automatically clean up expired events after the time expires, while simultaneously pushing notification information to the operations and maintenance platform via a message queue.

[0051] This technology effectively prevents the unlimited growth of the data synchronization queue by automatically clearing backlogged events, avoiding system resource exhaustion and accumulated synchronization delays caused by event accumulation. When system processing capacity is insufficient or network anomalies cause event backlog, a compensation mechanism can be triggered promptly to ensure the continuity of the synchronization process and data integrity. This significantly reduces the need for manual intervention and enhances the system's self-healing capabilities, thereby ensuring stable operation with strong data consistency in cloud database and search engine integration scenarios.

[0052] In a specific use case, taking the batch insertion of vehicle pass data as an example, the code flow in Outbox mode is illustrated as follows: Step 1: Batch write vehicle data to the table and add an Outbox, for example, insert 5 data entries: Algorithm 1 Batch Insert Vehicle Pass Data with Outbox Pattern Require:vehicleList:List of vehicle pass data Ensure: Consistent storage in business table and outbox table 1:BEGIN TRANSACTION 2: conn ←getConnection() 3:conn.setAutoCommit(false) 4:vehicleStmt←prepareStatement( 5: "INSERT INTO vehicle_pass_records VALUES(?,?,?,?...)") 6:outboxStmt←prepareStatement( 7: "INSERT INTO es-outboc VALUES(?,?,?,?)") 8:for each data in vehicleList do 9: / / Build business data batch 10:vehicleStmt.setParameters(data.id,data.cameraId,...) 11:vehicleStmt.addBatch() 12: / / Build outbox batch 13:payload←buildJsonPayload(data) 14:outboxStmt.setParameters("vehicle_pass_records", 15:data.id,"INSERT",payload) 16:outboxStmt.addBatch() 17: end for 18:vehicleStmt.erecuteBatch() 19:outboxStmt.executeBatch() 20: COMMIT TRANSACTION / / Generate gp_txid After execution, the Outbox table will look like this: 1--Vehicle Passage Record Sheet 2vehicle_pass_records: 5 records inserted 3 4--Outbox Table 5es_outbox: 6id |table_name| primary_key| op_type |gp_txid|status 71 |vehicle_pass_records|vP_001|I| 1001|0 82 |vehicle_pass_records|vP_002|I| 1001|0 93 |vehicle_pass_records|vP_003|I| 1001|0 104|vehicle_pass_records|vP_004|I| 1001|0 115|vehicle_pass_records|vP_005|I|1001|0 Step 2: Syncer acquires events and synchronizes them in batches to ES. Algorithm 1 Sync Vehicle Data to Elasticsearch 1: SCHEDULED (every 10s) 2: events ← query( 3: "SELECT * FROM es_outbox 4: WHERE status = 0 ORDER BY gp_trid 5: FOR UPDATE SKIP LOCKED LIMIT 1000") 6: bulkRequest ← new BulkRequest( 7: for each event in events do 8: Add IndexRequest with: 9:index: "vehicle_pass_index" 10:version:envent.gpTxid / / Key:use gp_txid as version 11:versionType: EXTERNAL 12: end for 13: response ← esClient.bulk(bulkRequest) 14: updateEventStatus(response,events) Step 3: Processing update messages When a message is updated, an event is inserted into the Outbox table at the same time as the business table is updated.

[0053] Algorithm 1 UpdateVehicle Status Require: vehicleId 1: BEGIN TRANSACTION 2:Execute:”UPDATE vehicle_pass_records 3:SET status =' PROCESSED' WHERE id =?" 4:Execute:"INSERT INTO es-outbox 5:(table_name,primary_key,op_type,payload) 6:VALUES(?,?,'UPDATE',?)" 7:COMMIT / / Generate new gp_txid = 1002 At this point, the current version of the document in Elasticsearch is version=1001. When updating gp_txid=1002, the document version is updated to 1002 because 1002>1001.

[0054] Step 4: Real-time detection and compensation Algorithm 1 Monitor Timeout Events 1: SCHEDULED (every 60s) 2:timeoutEvents ← query("SELECT * FROM es-outbox WHERE timeout") 3: if timeoutEvents ≠ 0 then 4:sendAlert(timeoutEvents.size()) 5: retrySync(timeoutEvents) 6:end if.

[0055] Please see Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the transaction synchronization method of this application. Specifically, it may include the following steps: Step S210: Save the transaction events written to the first system to the sending table.

[0056] This step is the same as step S110 above, and will not be repeated here.

[0057] Step S220: Batch push transaction events that meet the preset status requirements in the sending table to the second system.

[0058] This step is the same as step S120 above, and will not be repeated here.

[0059] Step S230: Verify the transaction event pushed to the second system. If the verification is successful, the transaction event is successfully pushed.

[0060] In some implementations, the integrity of the pushed transaction events can be verified, for example, by calculating the hash value of the event data (such as SHA-256) and comparing it with the hash value in the event header to determine whether the pushed transaction event is complete.

[0061] In addition, transaction events can be signed and verified. For example, transaction event data can be signed using encryption algorithms (such as HMAC and RSA). The second system verifies the signature to ensure that the event has not been tampered with and that its source is trustworthy.

[0062] Step S240: In response to successful push, update the status of the transaction event in the sending table.

[0063] This step is the same as step S130 above, and will not be repeated here.

[0064] This application uses the external version number of the second system as the transaction identifier for local transactions in the first system, and sets version_type=external in the second system to achieve idempotent writes. It also synchronously writes business transactions to the sending table and shares the same commit with the business table, thereby achieving strong consistency synchronization between the first and second systems and avoiding data inconsistency caused by dual writes. Combined with adaptive batch processing and exponential backoff retry mechanism, it significantly improves synchronization efficiency and reliability, and ensures timely recovery of data backlog through automatic compensation closed loop.

[0065] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0066] Please see Figure 3 , Figure 3 This is a schematic diagram of a framework of an embodiment of the transaction synchronization device 300 of this application. The transaction synchronization device 300 includes a saving module 310, a pushing module 320, and an updating module 330. The saving module 310 saves the transaction events written to the first system to a sending table, wherein the sending table is a database table used to store events to be synchronized, and the transaction events are generated based on local transactions in the first system. The pushing module 320 pushes the transaction events in the sending table that meet preset state requirements to the second system in batches. The updating module 330 updates the state of the transaction events in the sending table in response to successful push.

[0067] In some implementations, the push module 320 performs a preset state requirement that the transaction event is in a state to be synchronized, and pushes the transaction events in the sending table that meet the preset state requirement to the second system in batches, including: obtaining the number of events to be synchronized in the sending table, wherein the events to be synchronized are transaction events in the state to be synchronized; and in response to the number of events meeting the preset quantity requirement, pushing the events to be synchronized to the second system in batches.

[0068] In some implementations, the push module 320 performs a preset quantity requirement, including the number of events being less than a first preset value or the number of events being greater than a second preset value. In response to the number of events meeting the preset quantity requirement, the push module 320 pushes the events to be synchronized to the second system in batches, including: in response to the number of events being less than the first preset value, pushing the events to be synchronized in the sending table to the second system in batches, with a first quantity of events to be synchronized as a batch; or, in response to the number of events being greater than the second preset value, pushing the events to be synchronized in the sending table to the second system in batches, with a second quantity of events to be synchronized as a batch, wherein the second preset value is greater than the first preset value, and the second quantity is greater than the first quantity.

[0069] In some implementations, after the storage module 310 saves the transaction events written to the first system to the sending table, it includes: in response to the successful saving of the local transaction to the business table and the successful saving of the corresponding transaction event to the sending table, generating a transaction identifier for the local transaction, and storing the transaction identifier in the corresponding transaction event, wherein the business table is a database table in the first system that stores transaction events; after the transaction events in the sending table that meet the preset status requirements are pushed to the second system in batches, it includes: in response to the transactions in the second system meeting the version update requirements, updating the corresponding transaction events in the second system using the pushed transaction events, and using the transaction identifier as the version number of the updated transaction event.

[0070] In some implementations, the information of the transaction events in the sending table executed by the storage module 310 includes the business table name, business table primary key value, operation type, data content, transaction identifier, status, and creation time of the local transaction.

[0071] In some implementations, the update module 330 further includes the following steps: in response to a push failure, waiting for the current interval duration and then re-pushing the corresponding transaction event to the second system, wherein the current interval duration is the product of a preset interval duration and the number of re-pushing attempts; repeating the step of responding to a push failure, waiting for the current interval duration and then re-pushing the corresponding transaction event to the second system until the current interval duration is equal to or greater than a preset time threshold, stopping the push of the corresponding transaction event to the second system, and setting the status of the corresponding transaction event in the sending table to a failed state.

[0072] In some implementations, the update module 330 execution method further includes: obtaining the creation time data of transaction events in the sending table that are in a state of pending synchronization; deleting transaction events whose creation time data is greater than or equal to a preset creation time threshold, and providing corresponding prompts.

[0073] In some implementations, before the push module 320 pushes transaction events that meet the preset state requirements in the sending table to the second system in batches, it includes locking the transaction events that meet the preset state requirements in the sending table.

[0074] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the electronic device 40 of this application. The electronic device 40 includes a memory 41 and a processor 42 coupled to each other. The processor 42 is used to execute program instructions stored in the memory 41 to implement the steps in any of the above-described transaction synchronization method embodiments. In a specific implementation scenario, the electronic device 40 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 40 may also include mobile devices such as laptops and tablets, which are not limited here.

[0075] Specifically, processor 42 controls itself and memory 41 to implement the steps in any of the above-described transaction synchronization method embodiments. Processor 42 may also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 42 may be implemented using integrated circuit chips.

[0076] Please see Figure 5 , Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 50 of this application. The computer-readable storage medium 50 stores program instructions 501 that can be executed by a processor. The program instructions 501 are used to implement the steps in any of the above-described transaction synchronization method embodiments.

[0077] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0078] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A transaction synchronization method, characterized in that, include: The transaction events written to the first system are saved to the sending table, wherein the sending table is a database table used to store events to be synchronized, and the transaction events are generated based on local transactions in the first system; The transaction events that meet the preset status requirements in the sending table are pushed in batches to the second system; In response to a successful push, the status of the transaction event in the sending table is updated.

2. The method according to claim 1, characterized in that, The preset state requirement is that the transaction event is in a pending synchronization state. The step of batch pushing the transaction events in the sending table that meet the preset state requirement to the second system includes: Obtain the number of events to be synchronized in the sending table, wherein the events to be synchronized are transaction events in the state to be synchronized; In response to the number of events meeting the preset requirement, the events to be synchronized are pushed to the second system in batches.

3. The method according to claim 2, characterized in that, The preset quantity requirement includes the number of events being less than a first preset value or the number of events being greater than a second preset value. The step of responding to the event quantity meeting the preset quantity requirement by batch pushing the events to be synchronized to the second system includes: In response to the number of events being less than the first preset value, the events to be synchronized in the sending table are pushed to the second system in batches, with the first number of events to be synchronized forming a batch; or... In response to the number of events being greater than the second preset value, the events to be synchronized in the sending table are pushed to the second system in batches, with the second number of events to be synchronized as a batch, wherein the second preset value is greater than the first preset value and the second number is greater than the first number.

4. The method according to claim 1, characterized in that, After the transaction event written to the first system is saved to the sending table, the following steps are included: In response to the successful saving of the local transaction to the business table and the successful saving of the corresponding transaction event to the sending table, a transaction identifier for the local transaction is generated, and the corresponding transaction event stores the transaction identifier. The business table is a database table in the first system that stores the transaction events. After the transaction events that meet the preset status requirements in the sending table are pushed in batches to the second system, the process includes: In response to a transaction in the second system meeting the version update requirement, the corresponding transaction event in the second system is updated using the pushed transaction event, and the transaction identifier is used as the version number of the updated transaction event.

5. The method according to claim 1, characterized in that, The information of the transaction event in the sending table includes the business table name, business table primary key value, operation type, data content, transaction identifier, status, and creation time of the local transaction.

6. The method according to claim 1, characterized in that, The method further includes: In response to a push failure, after waiting for the current interval duration, the corresponding transaction event is re-pushed to the second system, wherein the current interval duration is the product of a preset interval duration and the number of re-pushing attempts; Repeat the steps of responding to the push failure, waiting for the current interval, and then re-pushing the corresponding transaction event to the second system until the current interval is equal to or greater than a preset time threshold, then stop pushing the corresponding transaction event to the second system, and set the status of the corresponding transaction event in the sending table to the failure status.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the creation time data of transaction events in the sending table that are in the state of pending synchronization; Delete transaction events whose creation time data is greater than or equal to a preset creation time threshold, and provide a corresponding prompt.

8. The method according to claim 1, characterized in that, Before the transaction events that meet the preset status requirements in the sending table are pushed to the second system in batches, the following steps are included: Lock the transaction events in the sending table that meet the preset state requirements.

9. An electronic device, characterized in that, It includes a memory and a processor that are coupled to each other, the processor being used to execute program instructions stored in the memory to implement the transaction synchronization method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the transaction synchronization method according to any one of claims 1 to 8.

Images