Data synchronization method and related device

By reading task objects from the transaction table and using the target adapter to write the data of each node of the transaction to the data source all at once, the problem of low transaction data synchronization efficiency in microservice architecture is solved, achieving efficient data synchronization and consistency, and supporting smooth database upgrades.

CN115587145BActive Publication Date: 2026-11-10CHINA CONSTRUCTION BANK +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211178596.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-26
Publication Date
2026-11-10
Estimated Expiration
2042-09-26

AI Technical Summary

Technical Problem

In a microservice architecture, the synchronization efficiency of transactional data is low, especially in a multi-node environment. Existing technologies rely on triggers, which leads to low synchronization efficiency.

Method used

Read the task object from the transaction log, determine the target adapter, and write the data objects generated by each node of the transaction to the target data source all at once through the target adapter, avoiding the need for triggers and using the adapter to achieve data synchronization.

Benefits of technology

It improves the efficiency of transaction data synchronization, ensures data consistency, supports a smooth transition for database upgrades, and reduces the complexity of the synchronization process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115587145B_ABST
    Figure CN115587145B_ABST
Patent Text Reader

Abstract

This invention provides a data synchronization method and related equipment, belonging to the field of data synchronization technology. The method includes: reading a task object from a transaction table. The task object includes a first synchronization object, which comprises multiple data objects and corresponding operation methods. Each data object represents data generated by different nodes of a transaction. The operation methods instruct the data source to perform data synchronization operations corresponding to the operation methods on the data objects. A target adapter is determined based on the task object, and the target adapter is controlled to perform a write operation on the first synchronization object in the task object, thereby writing the first synchronization object to the data source corresponding to the target adapter. In this invention, the synchronization objects in the task object include data generated by each node of the transaction and their operation methods, meaning that the data generated by each node of the transaction is synchronized to the database at once, improving the synchronization efficiency of transaction data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data synchronization technology, and in particular to a data synchronization method and related equipment. Background Technology

[0002] In a microservice architecture, each service has its own data storage method. To ensure the reliability of the entire architecture, some basic data (such as user information, organizational information, etc.) will be kept as copies in the database corresponding to each service, that is, data will be backed up synchronously in the database.

[0003] A transaction has multiple nodes, each corresponding to a service in a microservice architecture. For example, a transaction involving a transfer includes multiple nodes such as the transferor's account input, the transfer amount, and the amount received.

[0004] In the exemplary technology, data synchronization in the database is triggered by triggers; that is, if a node of a transaction is detected, the data generated by that node is synchronized to the database. However, since a transaction has multiple nodes, multiple node devices need to be synchronized to the database sequentially, resulting in low synchronization efficiency of transaction data. Summary of the Invention

[0005] This invention provides a data synchronization method and related equipment to solve the problem of low synchronization efficiency of transaction data.

[0006] On one hand, the present invention provides a data synchronization method, comprising:

[0007] Read the task object from the transaction table. The task object includes a first synchronization object. The first synchronization object includes multiple data objects and corresponding operation methods. Each data object is data generated by a different node of the transaction. The operation method is used to instruct the data source to perform the data synchronization operation corresponding to the operation method on the data object.

[0008] The target adapter is determined based on the task object, and the target adapter is controlled to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter.

[0009] In one embodiment, the target adapter determined based on the task object includes:

[0010] Obtain the description information from the task object, and determine the target data source to be written to the first synchronization object based on the description information;

[0011] The adapter corresponding to the target data source is determined as the target adapter corresponding to the task object.

[0012] In one embodiment, after obtaining the description information from the task object, the method further includes:

[0013] If the data source to be written to the first synchronization object cannot be determined based on the description information, a prompt message is output, and the description information in the task object is retrieved again after a target time interval. The prompt message is used to indicate that the first synchronization object has not specified a data source.

[0014] In one embodiment, the output prompt information includes:

[0015] If the data source to be written to the first synchronization object cannot be determined based on the description information, update the number of times the first synchronization object repeatedly performs the write operation;

[0016] If the preset number of attempts is not reached, the prompt message will be output.

[0017] In one embodiment, after updating the number of times the first synchronization object is repeatedly executed for write operations, the method further includes:

[0018] Once the preset number of attempts is reached, the task object is discarded.

[0019] In one embodiment, before reading the task object from the transaction table, the method further includes:

[0020] When the transaction is detected, the data objects generated by each node of the transaction are obtained;

[0021] The operation method of the data object is determined based on the node;

[0022] A first synchronization object is generated based on each of the data objects and their corresponding operation methods, and a task object is created based on the first synchronization object;

[0023] Insert the task object into the transaction table.

[0024] In one embodiment, creating a task object based on the first synchronization object includes:

[0025] When the operation mode of the data object is modification, the second synchronization object of the previous transaction is obtained, and the task object is generated based on the first synchronization object and the second synchronization object;

[0026] The step of controlling the target adapter to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter, includes:

[0027] The target adapter is controlled to perform write operations on the first synchronization object and the second synchronization object of the task object, so as to write the first synchronization object and the second synchronization object to the data source corresponding to the target adapter.

[0028] On the other hand, the present invention also provides a data synchronization device, comprising:

[0029] A reading module is used to read task objects from a transaction table. The task objects include a first synchronization object, which includes multiple data objects and corresponding operation methods. Each data object is data generated by a different node of a transaction. The operation method is used to instruct the data source to perform a data synchronization operation corresponding to the operation method on the data object.

[0030] The determination module is used to determine the target adapter based on the task object, and control the target adapter to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter.

[0031] On the other hand, the present invention also provides a data synchronization device, including: a memory and a processor;

[0032] The memory stores computer-executed instructions;

[0033] The processor executes computer execution instructions stored in the memory, causing the processor to perform the data synchronization method described above.

[0034] On the other hand, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the data synchronization method described above.

[0035] On the other hand, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data synchronization method described above.

[0036] The data synchronization method and related equipment provided by this invention read task objects from a transaction log, determine the target data source based on the task objects, and then control the target adapter to write the synchronization objects in the task objects to the corresponding data source of the target adapter. In this invention, the synchronization objects in the task objects include the data and operation methods generated by each node of the transaction, that is, the data generated by each node of the transaction is synchronized to the database at once, which improves the synchronization efficiency of transaction data. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0038] Figure 1 This is a schematic diagram of nodes in the first embodiment of the data synchronization method of the present invention;

[0039] Figure 2 This is a flowchart illustrating the second embodiment of the data synchronization method of the present invention;

[0040] Figure 3 This is a flowchart illustrating the third embodiment of the data synchronization method of the present invention;

[0041] Figure 4 This is a flowchart illustrating the fourth embodiment of the data synchronization method of the present invention;

[0042] Figure 5 This is a simplified flowchart of the data synchronization method of the present invention;

[0043] Figure 6 This is a schematic diagram of the data synchronization device of the present invention;

[0044] Figure 7 This is a schematic diagram of the hardware structure of the data synchronization device of the present invention.

[0045] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0047] It should be noted that the data involved in this invention is data authorized by the user or fully authorized by all parties. The collection, dissemination, and use of data in this invention all comply with the requirements of relevant national laws and regulations.

[0048] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0049] Reference Figure 1 , Figure 1 This is a first embodiment of the data synchronization method of the present invention. The data synchronization method includes the following steps:

[0050] Step S101: Read the task object from the transaction table. The task object includes a first synchronization object, which includes multiple data objects and corresponding operation methods. Each data object is data generated by different nodes of the transaction. The operation method is used to instruct the data source to perform the data synchronization operation corresponding to the operation method on the data object.

[0051] In this embodiment, the execution entity is a data synchronization device. For ease of description, the term "device" will be used to refer to the data synchronization device below. The device can be any terminal device with data processing capabilities.

[0052] The transaction log contains multiple task objects, each including a synchronization object, defined as the first synchronization object. The first synchronization object includes operation methods corresponding to one or more data objects. If a transaction has only one node, the synchronization object includes one data object; if a transaction has multiple nodes, the synchronization object includes the data objects corresponding to each node. Data objects are the data generated by the nodes of a transaction. Operation methods are used by the data source to perform corresponding data synchronization operations on the data objects. Operation methods include, for example, modification, replacement, addition, and deletion operations.

[0053] It should be noted that a task object includes one or more synchronization objects. For example, one synchronization object includes data objects that are currently generated by each node of the transaction, while another synchronization object includes data objects that are previously generated by each node of the transaction.

[0054] Step S102: Based on the target adapter determined by the task object, control the target adapter to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter.

[0055] The device is equipped with multiple types of adapters, including DB (DoggaByte, data storage unit) adapters and LDAP (Lightweight Directory Access Protocol) adapters. These adapters are used to write synchronization objects to their corresponding data sources. Data sources include DB data sources and LDAP data sources. A data source refers to the database or database server used by the database application.

[0056] The device determines the target adapter based on the task object. Specifically, the task object includes specified information, such as the data source for the first synchronization object. The device determines the data source for storing the first synchronization object based on the specified information in the task object, and the adapter corresponding to this data source is then identified as the target adapter. The device then controls the target adapter to perform a write operation on the first synchronization object, thereby writing the first synchronization object to the data source corresponding to the target adapter, thus completing the synchronization of data generated by each node in the transaction.

[0057] Furthermore, the flow table includes flags indicating whether task objects can be processed in parallel or must be processed serially; that is, each task object has a corresponding flag. The device includes serial and parallel executors. The serial executor reads task objects marked for serial processing from the flow table and sends them to the corresponding target adapter. The parallel executor reads task objects marked for parallel processing and synchronously sends each parallel processing task object to its corresponding target adapter for parallel processing.

[0058] In existing technologies, the transaction log formats in databases such as Oracle and MySQL are proprietary. When these databases are upgraded, their transaction log formats also need to be upgraded. If data synchronization is required, the database upgrade needs to be paused to perform the synchronization, resulting in poor stability during the upgrade process. In this embodiment, by having the adapter write the data to be synchronized into the data source, data synchronization can be achieved without interrupting the database upgrade. This allows for a smooth transition during the database upgrade and increases the flexibility of data synchronization.

[0059] Furthermore, in existing technologies, data synchronization relies on triggers. Different databases have different syntax for triggers, but data synchronization is performed between primary and backup databases. Because of these different syntaxes, the data formats in the primary and backup data sources differ, making it impossible to guarantee data consistency. In this embodiment, data synchronization does not rely on triggers, avoiding the possibility of the same data being stored in different formats in different databases, thus ensuring data consistency.

[0060] In this embodiment, the task object is read from the transaction table, and the target data source is determined based on the task object. Then, the target adapter is controlled to write the synchronization object in the task object to the data source corresponding to the target adapter. In this invention, the synchronization object in the task object includes the data and operation methods generated by each node of the transaction. That is, the data generated by each node of the transaction is synchronized to the database at once, which improves the synchronization efficiency of transaction data.

[0061] Reference Figure 2 , Figure 2 This is a second embodiment of the data synchronization method of the present invention. Based on the first embodiment, step S102 includes:

[0062] Step S201: Obtain the description information from the task object.

[0063] Step S202: Determine the target data source to be written to the first synchronization object based on the description information.

[0064] In this embodiment, the task object is a self-describing object that includes all the information required for data synchronization, including descriptive information. This descriptive information indicates the data source to which the synchronization object within the task object is to be written.

[0065] The description information includes the target data source to which the first synchronization object is to be written, or the device can determine the target data source to which the first synchronization object is to be written based on the description of the task object in the description information. For example, if the description information indicates that the data of the first synchronization object is data generated by the LDAP protocol, then the device can determine that the data source to which the first synchronization object is to be written is an LDAP data source based on the description information.

[0066] Step S203: Determine the adapter corresponding to the target data source as the target adapter corresponding to the task object.

[0067] In response, the device obtains description information from the task object, and then determines the target data source to be written to the first synchronization object based on the description information.

[0068] Furthermore, an OP (operation mode) can specify one or more adapters, meaning the data source corresponding to each adapter supports the operation mode for the data object. After determining the data source to be written to the first synchronization object based on the description information, the device then determines whether the data source to be written supports the operation mode corresponding to the first synchronization mode. If it does, the data source to be written is designated as the target data source. If it does not support it, the writing to the first synchronization object is paused, and a prompt to modify the data source to be written is displayed, before the writing of the first synchronization object is attempted again.

[0069] In this embodiment, the device determines the target data source to be written to the first synchronization object by using the description information of the task object, thereby accurately determining the adapter corresponding to the task object based on the adapter corresponding to the target data source.

[0070] Reference Figure 3 , Figure 3 This is a third embodiment of the data synchronization method of the present invention. Based on the second embodiment, after step S201, it further includes:

[0071] Step S301: When the data source to be written to the first synchronization object cannot be determined based on the description information, a prompt message is output, and the description information in the task object is retrieved again after a target time interval. Then, the step of determining the target data source to be written to the first synchronization object based on the description information is returned. The prompt message is used to indicate that the first synchronization object has not specified a data source.

[0072] In this embodiment, if the description information does not include a specified target data source and the device cannot determine the data source to be written to the first synchronization object based on the description information, the writing to the first synchronization object is terminated. Furthermore, the device outputs a prompt message indicating that the first synchronization object has not specified a data source, i.e., that the first synchronization object cannot be written to, thereby prompting the user to add the data source to be written to the first synchronization object in the description information.

[0073] The device will then output a prompt message and will retrieve the description information from the task object again at the target time interval, that is, retry at the target time interval to write the first synchronization object to the corresponding data source.

[0074] It should be noted that the executor is configured with a file. If the file includes a retry field, and the device cannot determine the first synchronous data source to be written based on the description information, the device will output a prompt message and retrieve the description information from the task object again at the target interval. For example, if the file includes a destroy field, the device can determine that the executor is configured with a retry mechanism.

[0075] In addition, the number of retries and the target duration of each retrieval can be configured via a file. The number of retries is stored in the device as a preset number. When the device cannot determine the data source to be written to the first synchronization object based on the description information, the device updates the number of times the first synchronization object will repeatedly perform the write operation, i.e., determines the number of retries. Each time the device cannot determine the data source to be written to the first synchronization object based on the description information, it increases this number. If the number of retries has not been reached, a prompt message is output, indicating that retries for writing to the first synchronization object are allowed. When the preset number of retries is reached, the task object is discarded.

[0076] In addition, the task object is set with a flag. If the device determines that the first synchronization object is being written serially based on the flag, the target duration is a fixed duration. If the first synchronization object is being written in parallel, the target duration is obtained by exponentially increasing the fixed duration. For example, if a first retry is performed, the target duration is the square of the fixed duration; if a second retry is performed, the target duration is the cube of the fixed duration.

[0077] It should be noted that when the executor malfunctions, the device cannot write to the synchronization object. That is, after identifying the target adapter, if the device detects an executor malfunction, it will re-obtain the description information from the task object at target intervals to repeat the writing of the synchronization object. Examples of executor malfunctions include: the database for the transaction table is not connected, or the executor initialization fails.

[0078] In this embodiment, when the device cannot determine the data source to be written to the first synchronization object based on the description information, it outputs a prompt and obtains the description information of the task object at target intervals, thereby repeatedly writing to the first synchronization object.

[0079] Reference Figure 4 , Figure 4 In the fourth embodiment of the data synchronization method of the present invention, based on any one of the first to third embodiments, before step S101, the method further includes:

[0080] Step S401: When a transaction is detected, obtain the data objects generated by each node of the transaction.

[0081] In this embodiment, when the application generates a transaction, that is, when the device detects the business logic, the device can determine that a transaction has occurred, and the device then obtains the data objects generated by each node of the transaction.

[0082] Step S402: Determine the operation method of the data object based on the node.

[0083] The device determines the operation method for data objects based on nodes. Specifically, different nodes require different methods of data synchronization. For example, if the node is the account selection in a transfer, and the selected account is a new account, the operation method is to add it, that is, the database adds the new account to the transfer account list corresponding to the account holder.

[0084] Step S403: Generate a first synchronization object based on each data object and its corresponding operation method, and create a task object based on the first synchronization object.

[0085] Step S404: Insert the task object into the log table.

[0086] The device generates a first synchronization object based on each data object and its corresponding operation method, and then creates a task object based on the first synchronization object, that is, adds the first synchronization object to the task object. The device then inserts the task object into the transaction table.

[0087] In this embodiment, the device has an API (Application Program Interface) interface. The API interface generates task objects, and the device inserts these task objects into the transaction log through the API interface. Furthermore, the API interface receives data from each node of the transaction. Specifically, the API interface is the only interface for interaction with external systems. These external systems can be applications stored within the device. After detecting a transaction, the application fills in tables based on the data generated by the nodes of the transaction, and the tables filled in by data from different nodes are different. For example, a transaction may have two nodes: the data from the first node is written to table A, and the data from the second node is written to table B. Operation method A for table A and operation method B for table B are cached. When the transaction ends, the data generated by all nodes of the transaction is written to different tables. The interceptor in the device intercepts all tables and their corresponding insertion operations, and then sends each table and its corresponding insertion operation to the API interface.

[0088] The API interface creates task objects using static methods, and then creates synchronization objects based on the data in each table and the corresponding operations. In other words, the synchronization objects include all the data generated by each node and their corresponding operations. The API interface then pushes the task objects to a task queue, which acts as a log table. It's important to note that different applications insert data into the API interface in different ways. For example, data can be inserted into the API interface via JDBC or Spring.

[0089] The transaction table includes various flags to indicate whether parallel processing is possible (or must be serial), or which sources have been successfully synchronized. It also contains all the information and parameters required for a single operation (e.g., writing one record to table A, then writing one record to table B). The following example illustrates the structure of the transaction table; refer to Table-1 for details:

[0090] Table 1

[0091]

[0092]

[0093] based on Figure 5 Here is a brief explanation of the data synchronization in this embodiment:

[0094] ① Synchronization API creates task objects. The ODS (Operational Data Store) synchronization program inserts data into the synchronization API via JDBC, while the application inserts data into the synchronization API via Spring. The synchronization API then creates task objects based on the data. The synchronization API supports a task object containing one or more synchronization objects, and a synchronization object can contain both new and old objects. These new and old objects facilitate data synchronization between the database and the database.

[0095] ② Synchronous API Insertion of Tasks (Task Objects). The synchronous API inserts the created task objects into the synchronous pipeline table. A synchronous pipeline table means that all task objects in the pipeline table require data synchronization. The synchronous pipeline table supports inserting task objects using Spring, JDBC, and other methods, and supports inserting both sequential and parallel task objects.

[0096] ③ Reading Records. Reading records refers to the serial executor reading serial task objects from the synchronous pipeline and the parallel executor reading parallel task objects from the synchronous pipeline. Furthermore, the serial and parallel executors are configured with a maximum number of retries, a retry interval, whether to automatically discard task objects, and support for one OP to specify one or more adapters.

[0097] ④ Parse the task and send it to the adapter. Parallel and serial executors parse the task object and then send it to the corresponding adapter. Adapters include DB adapters, LDAP adapters, and other adapters.

[0098] ⑤ The adapter writes data to the data source. For example, the DB adapter writes data to the DB data source. Data writing operations can be performed using a thread pool. The thread pool supports automatic synchronization after WebLogic starts, simultaneous execution of serial and parallel executors, simultaneous execution of multiple threads in the parallel executor, setting a delayed start time, and setting the maximum number of parallel threads.

[0099] In existing technologies, data is injected into the data source through Spring implementation classes, resulting in low data synchronization efficiency. This embodiment improves data synchronization efficiency by using a connection pool.

[0100] Furthermore, the API interface generates task objects, which is actually the API interface calling the application using an SDK (Software Development Kit). This solves problems such as multiple data sources, data consistency, and data synchronization errors in data synchronization, and also shields the complexity of the underlying implementation of data synchronization, reducing the application's data synchronization process and allowing the application to focus on processing business logic.

[0101] In this embodiment, when the device detects that a transaction has occurred, it obtains the data objects generated by each node of the transaction, determines the operation mode of the data objects according to the nodes, generates a first synchronization object based on each data object and the corresponding operation mode, creates a task object based on the first synchronization object, and finally inserts the task object into the transaction table so that the data of each node of the transaction can be synchronized.

[0102] In one embodiment, when the operation mode of the data object is modification, the device obtains the second synchronization object from the previous transaction, and then generates a task object based on the second synchronization object and the first synchronization object. For example, the first synchronization object can be placed at the beginning of the string corresponding to the task object, and the second synchronization object can be placed at the end of the string corresponding to the task object. The task object is, for example, (task, add("op", user)task, add("op", user, oidUser)), where op is the operation mode, user is the first synchronization object, and olduser is the second synchronization object.

[0103] The device controls the target adapter to perform write operations on the first synchronization object and the second synchronization object of the task object, thereby writing the first synchronization object and the second synchronization object to the data source corresponding to the target adapter, so that the data source writes the newly generated data of the transaction to the data source based on the first synchronization object and the second synchronization object.

[0104] The present invention also provides a data synchronization device, with reference to Figure 6 The data synchronization device 600 includes:

[0105] The reading module 610 is used to read task objects from the transaction table. The task objects include a first synchronization object, which includes multiple data objects and corresponding operation methods. Each data object is data generated by different nodes of the transaction. The operation method is used to instruct the data source to perform data synchronization operations corresponding to the operation method on the data objects.

[0106] The control module 620 is used to determine the target adapter based on the task object and control the first synchronization object in the target adapter's task object to perform a write operation, so as to write the first synchronization object to the data source corresponding to the target adapter.

[0107] In one embodiment, the data synchronization device 600 includes:

[0108] The acquisition module is used to acquire the description information in the task object and determine the target data source to be written to the first synchronization object based on the description information.

[0109] The determination module is used to determine the adapter corresponding to the target data source as the target adapter corresponding to the task object.

[0110] In one embodiment, the data synchronization device 600 includes:

[0111] The output module is used to output a prompt message when the data source to be written to the first synchronization object cannot be determined based on the description information, and to retrieve the description information from the task object again at a target time interval. The prompt message is used to indicate that the first synchronization object has not specified a data source.

[0112] In one embodiment, the data synchronization device 600 includes:

[0113] The determination module is used to update the number of times the write operation is repeatedly performed on the first synchronization object when the data source to be written to the first synchronization object cannot be determined based on the description information.

[0114] The output module is used to output a prompt message if the preset number of attempts has not been reached.

[0115] In one embodiment, the data synchronization device 600 includes:

[0116] The control module 620 is also used to discard the task object when the preset number of attempts is reached.

[0117] In one embodiment, the data synchronization device 600 includes:

[0118] The acquisition module is used to acquire the data objects generated by each node of a transaction when a transaction is detected.

[0119] The determination module is used to determine the operation method of data objects based on nodes;

[0120] The production module is used to generate a first synchronization object based on each data object and its corresponding operation method, and to create a task object based on the first synchronization object.

[0121] The Insert module is used to insert task objects into the pipeline.

[0122] In one embodiment, the data synchronization device 600 includes:

[0123] The acquisition module is used to acquire the second synchronization object from the previous transaction when the operation mode of the data object is modification, and to generate a task object based on the first synchronization object and the second synchronization object;

[0124] The control module 620 is also used to control the target adapter to write the first synchronization object and the second synchronization object of the task object into the data source corresponding to the target adapter.

[0125] Figure 7 This is a hardware structure diagram of a data synchronization device according to an exemplary embodiment.

[0126] The data synchronization device 700 may include: a processor 701, such as a CPU, a memory 702, and a transceiver 703. Those skilled in the art will understand that... Figure 7 The structure shown does not constitute a limitation on the data synchronization device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Memory 702 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0127] The processor 701 can call the computer program stored in the memory 702 to complete all or part of the steps of the above data synchronization method.

[0128] Transceiver 703 is used to receive information sent by external devices and to send information to external devices.

[0129] A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a data synchronization device, the data synchronization device is able to perform the aforementioned data synchronization method.

[0130] A computer program product includes a computer program that, when executed by a processor of a data synchronization device, enables the data synchronization device to perform the aforementioned data synchronization method.

[0131] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0132] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A data synchronization method, characterized in that, include: Read task objects from the transaction log. The task objects include a first synchronization object, which includes multiple data objects and corresponding operation modes. Each data object is data generated by a different node of the transaction. The operation mode is used to instruct the data source to perform data synchronization operation corresponding to the operation mode on the data object. The task objects in the transaction log have flags indicating parallel or serial processing. The target adapter is determined based on the task object, and the target adapter is controlled to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter; Before reading the task object from the transaction table, the process also includes: When the transaction is detected, the data objects generated by each node of the transaction are obtained through the API interface based on each table. The table stores the data generated by all nodes of the transaction. The data is filled into the table by the application based on the data generated by the nodes of the transaction after the transaction is detected. The tables filled with data from different nodes are different. The operation method of the data object is determined based on the node; A first synchronization object is generated based on each of the data objects and their corresponding operation methods, and a task object is created based on the first synchronization object; Insert the task object into the transaction table; The step of creating a task object based on the first synchronization object includes: When the operation mode of the data object is modification, the second synchronization object of the previous transaction is obtained, and the task object is generated based on the first synchronization object and the second synchronization object; The step of controlling the target adapter to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter, includes: The target adapter is controlled to perform write operations on the first synchronization object and the second synchronization object of the task object, so as to write the first synchronization object and the second synchronization object to the data source corresponding to the target adapter; The target adapter determined based on the task object includes: Obtain the description information from the task object, and determine the target data source to be written to the first synchronization object based on the description information; The adapter corresponding to the target data source is determined as the target adapter corresponding to the task object.

2. The data synchronization method according to claim 1, characterized in that, After obtaining the description information from the task object, the process further includes: If the data source to be written to the first synchronization object cannot be determined based on the description information, a prompt message is output, and the description information in the task object is retrieved again after a target time interval. The prompt message is used to indicate that the first synchronization object has not specified a data source.

3. The data synchronization method according to claim 2, characterized in that, The output prompt information includes: If the data source to be written to the first synchronization object cannot be determined based on the description information, update the number of times the first synchronization object repeatedly performs the write operation; If the preset number of attempts is not reached, the prompt message will be output.

4. The data synchronization method according to claim 3, characterized in that, After updating the number of repeated write operations on the first synchronization object, the method further includes: Once the preset number of attempts is reached, the task object is discarded.

5. A data synchronization device, characterized in that, include: A reading module is used to read task objects from a transaction table. The task objects include a first synchronization object, which includes multiple data objects and corresponding operation modes. Each data object is data generated by a different node of a transaction. The operation mode is used to instruct the data source to perform a data synchronization operation corresponding to the operation mode on the data object. The task objects in the transaction table have a flag indicating parallel or serial processing. The determination module is used to determine the target adapter based on the task object, and control the target adapter to perform a write operation on the first synchronization object in the task object, so as to write the first synchronization object to the data source corresponding to the target adapter; The acquisition module is used to acquire data objects generated by each node of the transaction through an API interface based on each table when the transaction is detected. The table stores the data generated by all nodes of the transaction. The data is filled into the table by the application based on the data generated by the nodes of the transaction after the transaction is detected. The table filled with the data of different nodes is different. The determining module is used to determine the operation mode of the data object based on the node; The production module is used to generate a first synchronization object based on each of the data objects and the corresponding operation methods, and to create a task object based on the first synchronization object. An insertion module is used to insert the task object into the flow table; The acquisition module is used to acquire the second synchronization object of the previous transaction when the operation mode of the data object is modification, and generate the task object based on the first synchronization object and the second synchronization object; The control module is used to control the target adapter to perform write operations on the first synchronization object and the second synchronization object of the task object, so as to write the first synchronization object and the second synchronization object to the data source corresponding to the target adapter; The acquisition module is used to acquire the description information in the task object and determine the target data source to be written to the first synchronization object based on the description information. The determining module is used to determine the adapter corresponding to the target data source as the target adapter corresponding to the task object.

6. A data synchronization device, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the data synchronization method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data synchronization method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data synchronization method according to any one of claims 1-4.

Citation Information

Patent Citations

  • Data synchronization method and system for heterogeneous databases

    CN103136231A

  • Data synchronization method and device between databases, computer equipment and storage medium

    CN110704532A