Method and device for accelerating data synchronization

Abstract

The invention relates to the technical field of data synchronization, and provides a method and a device for accelerating data synchronization. According to the method, the current operation log obtained from the source end database and all the historical operation logs in the operation registration chain table are subjected to conflict check, and when at least one historical operation log conflicts with the current operation log, the current operation log is added into the operation registration chain table; establishing a conflict association relationship between the at least one historical operation log and the current operation log; and according to the conflict association relationship, the at least one historical operation log and the current operation log, generating a corresponding target fusion operation and executing the target fusion operation in a target database, and executing the target fusion operation in the target database instead of executing a plurality of data operations with conflicts in parallel in the target database. The problem that in the prior art, due to the fact that associated data operation cannot be executed in a multi-thread concurrent mode, the data synchronization speed is low is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a method and device for accelerating data synchronization. Background Art

[0002] With the rapid development of information construction, in order to meet the needs of business development, it is often necessary to synchronize the data of one database (ie, the source database) to another database (ie, the destination database) in real time.

[0003] In the process of data synchronization, in order to improve synchronization efficiency, multi-threaded concurrent execution is usually used to synchronize data operations. However, when using this multi-threaded concurrent execution method to perform data operations, the data operations must have no associated rows; multiple data operations with associated rows cannot be synchronized in multi-threaded concurrent mode, and must be executed strictly in the order of multiple data operations in the source database. For example, there is data operation 1 "insert a row R1 into database table T1", and data operation 2 "update the field value of R1"; at this time, the synchronization tool must first enable the target database to execute data operation 1 before executing data operation 2 to ensure data consistency between the source database and the target database.

[0004] In actual synchronization scenarios, when most data operations in the source database are related, most data operations can only be synchronized to the destination database serially, which greatly affects the data synchronization speed.

[0005] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method and device for accelerating data synchronization, the purpose of which is to perform conflict detection on the operation logs of the source database, generate corresponding target fusion operations according to multiple conflicting operation logs, and execute the target fusion operation in the target database instead of executing multiple data operations corresponding to the multiple conflicting operation logs in parallel in the target database, thereby solving the problem of slow data synchronization caused by the inability to execute multiple threads concurrently due to the existence of related data operations in the prior art.

[0007] The present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for accelerating data synchronization, comprising:

[0009] Obtain the current operation log from the source database, and perform a conflict check between the current operation log and all historical operation logs in the operation registration linked list;

[0010] When at least one historical operation log conflicts with the current operation log, the current operation log is added to the operation registration linked list to establish a conflict association relationship between the at least one historical operation log and the current operation log;

[0011] Generate a corresponding target fusion operation according to the conflict association relationship, the at least one historical operation log and the current operation log;

[0012] The target fusion operation is performed in the target database to complete data synchronization in the target database according to the at least one historical operation log and the current operation log.

[0013] Further, the obtaining of the current operation log from the source database and the performing of a conflict check between the current operation log and all the historical operation logs in the operation registration linked list include:

[0014] Obtain the current operation logs from the source database in ascending order of the source SCN;

[0015] For the current operation log, obtain the LSCN and ROWID of the current operation log; wherein the LSCN of the operation log is the time when the data operated by the operation log was last modified, and the ROWID of the operation log is the ROWID of the data operated by the operation log;

[0016] According to the order in which the historical operation logs are added to the operation registration linked list, the LSCN and ROWID of the historical operation logs are sequentially obtained from the operation registration linked list;

[0017] If the LSCN of the current operation log is inconsistent with the LSCN of the historical operation log, there is no conflict between the current operation log and the historical operation log; if the LSCN of the current operation log is consistent with the LSCN of the historical operation log, it is determined whether the ROWID of the current operation log is consistent with the ROWID of the historical operation log;

[0018] When the ROWID of the current operation log is consistent with the ROWID of the historical operation log, there is a conflict between the current operation log and the historical operation log, so as to complete the conflict check between the current operation log and the historical operation log.

[0019] Further, when there is a conflict between at least one historical operation log and the current operation log, adding the current operation log to the operation registration linked list, and establishing a conflict association relationship between the at least one historical operation log and the current operation log includes:

[0020] When the current operation log is added to the operation registration linked list, a conflict association relationship is established between each historical operation log and its next historical operation log in pairs according to the order in which the at least one historical operation log is added to the operation registration linked list;

[0021] A conflict association relationship is established between the last historical operation log in the at least one historical operation log and the current operation log.

[0022] Further, the generating a corresponding target fusion operation according to the conflict association relationship, the at least one historical operation log and the current operation log includes:

[0023] Obtaining the operation log corresponding to the transaction to be executed in the operation registration linked list;

[0024] When the acquired operation logs have conflicting association relationships, acquiring a conflicting association relationship sequence associated with the acquired operation logs; wherein a conflicting association relationship sequence includes at least one conflicting association relationship between the operation logs;

[0025] When the conflict association relationship of the obtained operation log is not the last conflict association relationship in the conflict association relationship sequence, ignoring the obtained operation log, and obtaining the next operation log of the obtained operation log in the to-be-executed transaction;

[0026] When the conflict association relationship of the acquired operation log is the last conflict association relationship in the conflict association relationship sequence, a target fusion operation corresponding to the conflict association relationship sequence is generated according to the operation logs in the conflict association relationship sequence.

[0027] Further, when the conflict association relationship of the acquired operation log is the last conflict association relationship in the conflict association relationship sequence, generating a target fusion operation corresponding to the conflict association relationship sequence according to the operation log in the conflict association relationship sequence includes:

[0028] Acquire the first historical operation log in the conflict association relationship sequence according to the conflict operation pointer; except for the first historical operation log, each operation log corresponding to the conflict association relationship sequence has a conflict operation pointer, and the corresponding conflict operation pointers all point to the first historical operation log;

[0029] The target fusion operation is generated according to the operation type and column value information of the first historical operation log and the current historical operation log.

[0030] Furthermore, generating the target fusion operation according to the operation type and column value information of the first historical operation log and the current historical operation log includes:

[0031] When the operation type of the first historical operation log is an insert operation, and the operation type of the current operation log is an update operation, the operation type of the target fusion operation is an insert operation; from the columns involved in all the old column value information of the current operation log, determine the columns not involved in all the new column value information of the current operation log, and obtain the old column value information to be inserted of the determined columns; based on all the new column value information of the current operation log and the old column value information to be inserted, generate the target fusion operation;

[0032] When the operation type of the first historical operation log is an insert operation, and the operation type of the current operation log is a delete operation, a target fusion operation is not generated, so that the target fusion operation is not subsequently executed in the target database;

[0033] When the operation type of the first historical operation log is an update operation, and the operation type of the current operation log is an update operation, the operation type of the target fusion operation is an update operation; according to all the old column value information of the first historical operation log and the current operation log, the column value information before the update and the column value information after the update are determined to generate the target fusion operation;

[0034] When the operation type of the first historical operation log is an update operation, and the operation type of the current operation log is a delete operation, the operation type of the target fusion operation is a delete operation; all old column value information of the first historical operation log is determined as the old column value information to be deleted, and the target fusion operation is generated according to the old column value information to be deleted.

[0035] Further, the determining the pre-update column value information and the post-update column value information according to all old column value information of the first historical operation log and the current operation log to generate the target fusion operation includes:

[0036] Using all old column value information of the first historical operation log as column value information before updating;

[0037] Determine, from the columns involved in the column value information before the update, the columns not involved in all the new column value information of the current operation log; obtain, from all the old column value information of the current operation log, the column value information to be updated of the determined columns;

[0038] Based on all new column value information of the current operation log and the column value information to be updated, obtaining updated column value information;

[0039] The target fusion operation is generated based on the pre-update column value information and the post-update column value information.

[0040] Furthermore, it also includes:

[0041] If the operation type of the current operation log is a delete operation, then selectively not generating a target fusion operation, or immediately generating a target fusion operation according to the current operation log and the corresponding first historical operation log, and executing the target fusion operation in the target database;

[0042] If the operation type of the current operation log is not a delete operation, the corresponding target fusion operation is not generated temporarily, and it is determined whether the conflict association relationship sequence corresponding to the current operation log increases within the first preset time;

[0043] When the corresponding conflict association relationship sequence is not increased within the first preset time, generating a target fusion operation according to the current operation log and the corresponding first historical operation log, and executing the target fusion operation in the target database;

[0044] When the corresponding conflict association relationship sequence increases within the first preset time, a corresponding target fusion operation is generated according to the last operation log in the increased conflict association relationship sequence and the corresponding first historical operation log, and the target fusion operation is executed in the target database.

[0045] In a second aspect, the present invention further provides a device for accelerating data synchronization, which is used to implement the method for accelerating data synchronization in the first aspect, and the device for accelerating data synchronization includes:

[0046] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the method for accelerating data synchronization described in the first aspect.

[0047] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method for accelerating data synchronization described in the first aspect.

[0048] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method for accelerating data synchronization as in the first aspect.

[0049] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer or a processor, enables the computer or the processor to execute the method for accelerating data synchronization as described in the first to fourth aspects and any one of the aspects.

[0050] In a sixth aspect, a system for accelerating data synchronization is provided, comprising the device for accelerating data synchronization as in the second aspect, and using the method for accelerating data synchronization as in the first aspect to complete the interaction of the device for accelerating data synchronization of the second aspect.

[0051] Different from the prior art, the present invention has at least the following beneficial effects:

[0052] The present invention performs a conflict check on the current operation log obtained from the source database and all the historical operation logs in the operation registration linked list. When at least one historical operation log conflicts with the current operation log, the current operation log is added to the operation registration linked list, and a conflict association relationship between at least one historical operation log and the current operation log is established; according to the conflict association relationship and the at least one historical operation log and the current operation log, a corresponding target fusion operation is generated and executed on the target database, so as to merge multiple data operations for the same row into one target fusion operation; by executing the target fusion operation on the target database, multiple conflicting data operations are executed in parallel on the target database instead of executing them in parallel, and the steps of data operations are reduced, so as to achieve faster synchronization efficiency, and solve the problem of slow data synchronization speed caused by the inability to execute multi-threaded concurrent execution of associated data operations in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 It is a flowchart of a method for accelerating data synchronization provided by an embodiment of the present invention;

[0055] Figure 2 It is a schematic diagram of an operation registration linked list and related data tables in a synchronization tool provided by an embodiment of the present invention;

[0056] Figure 3 is a flow chart of step 10 provided in an embodiment of the present invention;

[0057] Figure 4 It is a schematic diagram of a conflict check process provided by an embodiment of the present invention;

[0058] Figure 5 is a flow chart of step 20 provided in an embodiment of the present invention;

[0059] Figure 6 is a schematic diagram of an operation registration linked list provided by an embodiment of the present invention;

[0060] Figure 7 is a schematic diagram of a conflict association relationship sequence provided by an embodiment of the present invention;

[0061] Figure 8 is a flow chart of step 30 provided in an embodiment of the present invention;

[0062] Fig. 9 It is a schematic diagram of a conflict association relationship sequence after adding a conflict operation pointer provided by an embodiment of the present invention;

[0063] Fig.10 is a schematic diagram of another conflict association relationship sequence provided by an embodiment of the present invention;

[0064] Fig.11 It is a flowchart of a specific example of ignoring conflicting operation logs provided by an embodiment of the present invention;

[0065] Fig.12 is a flowchart of step 3042 provided by an embodiment of the present invention;

[0066] Fig.13 It is a flowchart of generating a target fusion operation under the situation of "INS+UPD" provided by an embodiment of the present invention;

[0067] Fig.14 It is a schematic diagram of a specific example of generating a target fusion operation under the situation of "INS+UPD" provided by an embodiment of the present invention;

[0068] Fig.15 is a flow chart of generating a target fusion operation under the situation of "UPD+UPD" provided by an embodiment of the present invention;

[0069] Fig.16 It is a schematic diagram of a specific example of generating a target fusion operation under the situation of "UPD+UPD" provided by an embodiment of the present invention;

[0070] Fig.17 It is a schematic diagram of a specific example of generating a target fusion operation under the situation of "UPD+DEL" provided by an embodiment of the present invention;

[0071] Fig.18 It is a schematic diagram of the overall process of a method for accelerating data synchronization provided by an embodiment of the present invention;

[0072] Fig.19 is a flow chart of another method for accelerating data synchronization provided by an embodiment of the present invention;

[0073] Fig. 20It is a schematic diagram of the architecture of a device for accelerating data synchronization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0075] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0076] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0077] In the description of the present invention, it is necessary to understand that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present disclosure.

[0078] In the description of the present invention, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, the same type of nouns may be described as two independent individuals by adding "A" and "B" at the end. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the same type of individuals for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0079] When describing some embodiments, the expressions "coupling", "coupling" and "connection" and their derivatives may be used. For example, when describing some embodiments, the term "connection" may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term "coupling" may be used to indicate that two or more components are in direct physical or electrical contact. However, the terms "connection" or "coupling" may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other, such as "optical path coupling", "wireless connection", etc. The embodiments disclosed herein are not necessarily limited to the contents of the present invention.

[0080] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) will be involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0081] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0082] Embodiment 1:

[0083] In order to solve the above problems, Figure 1 As shown, an embodiment of the present invention provides a method for accelerating data synchronization, including:

[0084] Step 10: Obtain the current operation log from the source database, and perform a conflict check between the current operation log and all historical operation logs in the operation registration linked list.

[0085] Among them, one source database corresponds to one operation registration linked list.

[0086] The database real-time synchronization tool mines the operation log of the source database, restores the data operations of the source database, and then synchronizes the data operations to the destination database. For ease of description, the following will use "synchronization tool" to refer to "database real-time synchronization tool". The smallest unit that the synchronization tool synchronizes between the source database and the destination database is the operation log, which contains the data rows of specific operations and modification records of the data rows. The synchronization tool parses the operation log of the source database, generates the operations performed in the destination database based on the information obtained from the analysis, and executes them in the destination database, thereby completing the synchronization of the corresponding data operations based on the operation log.

[0087] The synchronization tool first reads the operation logs of the source database in sequence, and adds the current operation log read to the operation registration list each time; when adding, the current operation log is detected for conflicts with the existing historical operation logs in the operation registration list. In an optional embodiment, the operation registration list can be implemented using a bidirectional linked list; a specific example of the operation registration list OPS_LST is as follows Figure 2 As shown, the operation registration list OPS_LST includes operation log OP1, operation log OP2 and operation log OP3, etc., and its structure and corresponding fields will be explained below.

[0088] Step 20: When there is a conflict between at least one historical operation log and the current operation log, the current operation log is added to the operation registration linked list, and a conflict association relationship between the at least one historical operation log and the current operation log is established.

[0089] Among them, the existence of a conflict means that the operation object of the corresponding historical operation log and the current operation log is the same row; for example, the historical operation log OP1 is the operation log of data operation 1 "insert a row R1 into database table T1" in the source database, the historical operation log OP2 is the operation log of data operation 2 "update the field value of R1 to 1" in the source database, and the current operation log OPn is the operation log of data operation n "update the field value of R1 to 4" in the source database, then the historical operation log OP1, the historical operation log OP2 and the current operation log OPn are in conflict. Each time the synchronization tool adds the current operation log it has read to the operation registration linked list, if it conflicts with the operation log already existing in the operation registration linked list, it is necessary to establish a corresponding conflict association relationship in the operation registration linked list to record the corresponding conflict, so as to facilitate the optimization of conflict handling during subsequent synchronization. For example, after performing a conflict check, the synchronization tool determines that the current operation log OPn conflicts with the historical operation logs OP1 and OP2 in the operation registration list OPS_LST, so the current operation log OPn is first added to the operation registration list OPS_LST, and the corresponding conflict association relationship is established when adding.

[0090] Step 30: Generate a corresponding target fusion operation according to the conflict association relationship, the at least one historical operation log and the current operation log.

[0091] For example, based on the historical operation log OP1, the information of data operation 1 is obtained, based on the historical operation log OP2, the information of data operation 2 is obtained, and based on the current operation log OPn, the information of data operation n is obtained. A target fusion operation is generated according to the above three information. Executing the target fusion operation in the target database can replace the serial execution of the historical operation log OP1, the historical operation log OP2 and the current operation log OPn.

[0092] Step 40: Execute the target fusion operation in the target database to complete data synchronization in the target database according to the at least one historical operation log and the current operation log.

[0093] The present invention performs a conflict check on the current operation log obtained from the source database and all the historical operation logs in the operation registration linked list. When at least one historical operation log conflicts with the current operation log, the current operation log is added to the operation registration linked list, and a conflict association relationship between at least one historical operation log and the current operation log is established; according to the conflict association relationship and the at least one historical operation log and the current operation log, a corresponding target fusion operation is generated and executed on the target database, so as to merge multiple data operations for the same row into one target fusion operation; by executing the target fusion operation on the target database, multiple conflicting data operations are executed in parallel on the target database instead of executing them in parallel, and the steps of data operations are reduced, so as to achieve faster synchronization efficiency, and solve the problem of slow data synchronization speed caused by the inability to execute multi-threaded concurrent execution of associated data operations in the prior art.

[0094] To illustrate the conflict checking process, Figure 3 As shown, the step 10 includes:

[0095] Step 101: Obtain the current operation logs from the source database in order from small to large SCN of the source.

[0096] Among them, the source SCN is the system change number (SCN) when the data operation corresponding to the operation log is executed in the source database. The source SCN is generated according to the internal clock of the source database. Since the source SCN of each data operation is from small to large when the source parses the corresponding data operation to generate the operation log, the OP_SCN (i.e., source SCN) in the operation registration list is also from small to large. Since multiple data operations with associated rows must be executed strictly according to the source SCN in the destination database, the synchronization tool reads the operation logs from the operation registration list in the order in which the operation logs are added to the operation registration list, generates the corresponding data operations, and synchronizes them to the destination database.

[0097] In an optional embodiment, the information obtained from the operation log may include: (1) OP_SCN: that is, the SCN of the source end. OP_SCN increases sequentially. The OP_SCN of the data operation executed later is greater than the OP_SCN of the data operation executed earlier; (2) OP_TRXID: the transaction number of the data operation. Each data operation includes a transaction number. Data operations with the same transaction number belong to the same transaction. (3) OP_OBJ: the object to which the data operation belongs; (4) OP_TYPE: the operation type of the data operation. The operation types include insert operation (INSERT), update operation (UPDATE), delete operation (DELELTE), transaction commit (COMMIT), and transaction rollback (ROLLBACK). (5) LSCN (Last System Change Number): SCN of the last operation on the data operated by this data operation; (6) OP_VAL: All column value information of the data operated by this data operation; among them, all column value information of the insert operation is all new column value information (OP_NEW_VAL), all column value information of the delete operation is all old column value information (OP_OLD_VAL), and all column value information of the update operation is all old column value information and all new column value information (OP_NEW_VAL+OP_OLD_VAL); (7) ROWID: The unique identifier of the row of the data operated by this data operation.

[0098] Step 102: For the current operation log, obtain the LSCN and ROWID of the current operation log; wherein the LSCN of the operation log is the time when the data operated by the operation log was last modified, and the ROWID of the operation log is the ROWID of the data operated by the operation log.

[0099] Among them, an operation log of a source database corresponds to a source SCN, the LSCN of the current operation log is the time when the data operated by the current operation log was last modified, and the ROWID of the current operation log is the ROWID of the data operated by the current operation log.

[0100] Step 103: according to the order in which the historical operation logs are added to the operation registration linked list, the LSCN and ROWID of the historical operation logs are sequentially obtained from the operation registration linked list.

[0101] The LSCN of the historical operation log is the time when the data operated by the historical operation log was last modified, and the ROWID of the historical operation log is the ROWID of the data operated by the historical operation log.

[0102] Step 104: If the LSCN of the current operation log is inconsistent with the LSCN of the historical operation log, there is no conflict between the current operation log and the historical operation log; if the LSCN of the current operation log is consistent with the LSCN of the historical operation log, determine whether the ROWID of the current operation log is consistent with the ROWID of the historical operation log.

[0103] If the LSCNs are consistent, it means that the corresponding data operations are performed on the same database table. Since the same LSCN may correspond to more than one data operation, it is necessary to further determine whether the ROWIDs are consistent.

[0104] Step 105: When the ROWID of the current operation log is consistent with the ROWID of the historical operation log, there is a conflict between the current operation log and the historical operation log, so as to complete the conflict check between the current operation log and the historical operation log.

[0105] For multiple operation logs operating the same database table, when the ROWIDs recorded therein are the same, it means that the multiple data operations operate on the same row of data, so there is a conflict between them. Figure 4 As shown, the LSCN of the current operation log is LSCN1. Each time, a LSCN of a historical operation log is obtained from the operation registration linked list OPS_LST, that is, SCNi, and LSCN1 is compared to see if it is equal to SCNi. If they are not equal, there is no conflict between the current operation log and the historical operation log obtained this time. If they are equal, continue to obtain the ROWID of the current operation log, that is, R1, and then obtain the ROWID of the historical operation log obtained this time, that is, R2. Compare whether R1 is equal to R2. If they are not equal, there is no conflict between the current operation log and the historical operation log obtained this time. If they are equal, there is a conflict between the current operation log and the historical operation log obtained this time.

[0106] In order to illustrate the process of establishing conflict associations, Figure 5 As shown, the step 20 includes:

[0107] Step 201: when the current operation log is added to the operation registration linked list, a conflict association relationship is established between each historical operation log and its next historical operation log in pairs according to the order in which the at least one historical operation log is added to the operation registration linked list.

[0108] Step 202: Establish a conflict association relationship between the last historical operation log in the at least one historical operation log and the current operation log.

[0109] When there is a conflict, it means that at least two data operations in the operation registration chain list operate on the same row of data, and it is necessary to associate the data operations that operate on the same row. Figure 6 As shown, for example, if there is a conflict between the historical operation log OP1 and the current operation log OPn, a conflict association relationship between the historical operation log OP1 and the current operation log OPn is established. In an optional embodiment, the conflict association relationship can be established using a pointer, and a pointer (op_next) is added to the historical operation log OP1, and the pointer points to the current operation log OPn with the conflicting operation.

[0110] like Figure 7 As shown, for multiple historical operation logs that conflict with the current operation log OPn, after the conflict association relationship is established, they will establish a series of conflict association relationships through the pointer op_next.

[0111] In order to explain the synchronization process of the synchronization tool, Figure 8 As shown, the step 30 includes:

[0112] Step 301: Obtain the operation log corresponding to the transaction to be executed in the operation registration linked list.

[0113] The synchronization tool synchronizes data from the source database to the destination database in transactions. Therefore, when synchronizing, the pending transaction is determined each time. During the synchronization of the pending transaction, the operation logs are taken from the operation registration list in turn, and synchronization is performed according to the corresponding operation logs.

[0114] The embodiment of the present invention provides a specific example of determining pending transactions and obtaining operation logs during synchronization, which is described below: Figure 2As shown, the synchronization tool synchronizes according to the pending transaction list TRX_LST; the pending transaction list TRX_LST is a collection of completed transactions in the source database. After parsing the operation log, the synchronization tool uses the transaction number OP_TRXID of the operation log as the key value to search for the corresponding transaction description information TRX_INFO in the synchronization tool transaction table TRX_HASH, and adds the operation log to TRX_OPS of TRX_INFO. Among them, TRX_HASH is a set of pointers pointing to TRX_INFO. TRX_INFO uses OP_TRXID as the key value, finds the corresponding position in TRX_HASH, and points the pointer of the corresponding position to TRX_INFO; TRX_OPS is a list of operation pointers contained in the transaction. TRX_INFO collects the operations of a transaction, and the synchronization tool synchronizes the operations to the destination database in units of transactions. If the operation type OP_TYPE of the parsed operation log is COMMIT or ROLLBACK, it indicates that the transaction has ended in the source database and the corresponding data can be synchronized to the destination database. At this time, TRX_OPS is removed from TRX_HASH and added to TRX_LST. In addition, you can also set up a transaction registration table TRX_TABLE. TRX_TABLE exists in the destination database. After the TRX_INFO of a transaction is executed in the destination database, the information of the transaction is registered in TRX_TABLE. TRX_TABLE includes the following fields: (1) TRXID: transaction number of TRX_INFO; (2) STARTSCN: SCN of the first operation log in TRX_INFO; (3) ENDSCN: SCN of the last operation log in TRX_INFO.

[0115] When there is no conflicting association relationship among the acquired operation logs, a corresponding operation statement is generated according to the acquired operation logs; the generated operation statement is executed in the target database to complete the synchronization of the acquired operation logs. The operation statement is implemented using Structured Query Language (SQL); for operation logs that do not have conflicts, the synchronization tool converts them into SQL statements and synchronizes them to the target database, and can use multi-threaded parallel synchronization.

[0116] Step 302: when there is a conflicting association relationship between the acquired operation logs, a conflicting association relationship sequence associated with the acquired operation logs is acquired; wherein a conflicting association relationship sequence includes at least one conflicting association relationship between the operation logs.

[0117] Combine the following Figure 6 and Figure 7, explain the relationship between the conflict association sequence and the conflict association:

[0118] like Figure 6 As shown, in the operation registration linked list OPS_LST, there are multiple operation logs in accordance with the source SCN sequence of reading the operation logs; among them, the historical operation log OP1 conflicts with the current operation log OPn, and there is a conflict association relationship between the two (that is, there is a pointer op_next in the historical operation log OP1, and it points to the current operation log OPn); and the historical operation log OP2 has no conflict with either the historical operation log OP1 or the current operation log OPn.

[0119] like Figure 7 The figure shows a specific example of a conflict association relationship sequence; the historical operation log OP1 conflicts with the historical operation log OPi, and there is a corresponding conflict association relationship between the two; the historical operation log OPi conflicts with the historical operation log OPj, and there is a corresponding conflict association relationship between the two; the historical operation log OPj conflicts with the current operation log OPn, and there is a corresponding conflict association relationship between the two; the historical operation log OP1, the historical operation log OPi, the historical operation log OPj and the current operation log OPn and the corresponding conflict association relationships between them constitute the following. Figure 7 A conflicting association relationship sequence is shown in FIG. Figure 6 The conflicting relationships and corresponding conflicting relationship sequences shown in Figure 7 Different than shown.

[0120] Step 303: when the conflict association relationship of the obtained operation log is not the last conflict association relationship in the conflict association relationship sequence, the obtained operation log is ignored, and the next operation log of the obtained operation log is obtained in the to-be-executed transaction.

[0121] like Figure 7 As shown, when processing the conflict between the historical operation log OP1, the historical operation log OPi, the historical operation log OPj and the current operation log OPn, the conflict association relationship sequence associated with the obtained operation logs is obtained; for example, when processing the historical operation log OP1, the conflict association relationship sequence associated with the obtained operation logs is obtained. Figure 7 The sequence of conflicting associations shown.

[0122] Step 304: when the conflict association relationship of the acquired operation log is the last conflict association relationship in the conflict association relationship sequence, a target fusion operation corresponding to the conflict association relationship sequence is generated according to the operation log in the conflict association relationship sequence.

[0123] In the prior art, the data operations corresponding to the conflicting operation logs need to be executed serially. Since the conflicting data operations process the same data row, during the synchronization process, only the final data operation result needs to be synchronized to the destination database. Therefore, during the synchronization process, the synchronization tool of the embodiment of the present invention adopts the method of "ignore + merge operation data" to optimize the synchronization of conflicting data. The specific process is described below: Step 304 includes:

[0124] Step 3041: Obtain the first historical operation log in the conflict association sequence according to the conflict operation pointer; except for the first historical operation log, each operation log corresponding to the conflict association sequence has a conflict operation pointer, and the corresponding conflict operation pointers all point to the first historical operation log.

[0125] like Fig. 9 As shown, the first historical operation log in the conflict association relationship sequence is the conflict operation head, and the last operation log in the conflict association relationship sequence is the conflict operation tail; when the current operation log OPn needs to be synchronized, since it is the conflict operation tail, the conflict operation head is determined based on the historical operation log OP1 pointed to by its conflict operation pointer op_base.

[0126] The conflict operation header and conflict operation pointer are explained below:

[0127] like Fig.10 As shown, the first conflict association relationship is represented by the pointer op_next from the operation log OP1 to the operation log OPi, and the conflict operation header is the operation log OP1. Fig. 9 As shown, when the corresponding conflict association relationship does not involve the conflict operation head, that is, for the operation log OPi, the operation log OPj and the operation log OPn, the conflict operation pointer op_base is constructed, so that op_base points to the operation log OP1; wherein the operation log OPn is the conflict operation tail. It should be noted that during the operation process, if the newly added operation log of a data row conflicts with the operation log in the operation registration linked list, then the corresponding conflict operation tail will change, that is, become the newly added operation log.

[0128] Step 3042: Generate the target fusion operation according to the operation type and column value information of the first historical operation log and the current historical operation log.

[0129] like Fig.11 As shown, when the synchronization tool processes the operation log according to the transaction, when processing any operation log except the tail of the conflicting operation, the corresponding operation log is ignored, that is, it is not converted into the corresponding SQL statement. Fig. 9As shown, when processing operation logs OP1, OPi, and OPj, they are all ignored; when processing operation log OPn, the target fusion operation is obtained based on the conflict operation fusion method for synchronization. The specific synchronization method will be described below. In actual application scenarios, the more conflicting conflict association relationship sequences there are, or the more operation logs in the conflict association relationship sequence, the more operation logs are ignored, and the faster the synchronization tool is improved.

[0130] No matter how many conflicting data operations a row of data corresponds to, the fusion only involves the first and last data operations in the conflicting relationship sequence. The corresponding operation logs are the first historical operation log OP1 and the current operation log OPn. This is because the first historical operation log OP1 contains the state of the data of the row in the destination database, and the current operation log OPn contains the final state of the data of the row after synchronizing each data operation in the conflicting relationship sequence. Since the operation type of the first historical operation log OP1 can only be an insert operation (as described below), Fig.14 INS) or an update operation (as shown in Fig.14 The operation type of the current operation log OPn can only be UPD or delete operation (as shown in the following Fig.15 DEL shown in ), so there are only four cases where there are conflicting operations in the fusion: "INS+UPD", "INS+DEL", "UPD+UPD" and "UPD+DEL".

[0131] In order to illustrate the process of generating the target fusion operation, as shown in Fig.12 As shown, the step 3042 includes:

[0132] Step 30421: When the operation type of the first historical operation log is an insert operation, and the operation type of the current operation log is an update operation, the operation type of the target fusion operation is an insert operation; from the columns involved in all the old column value information of the current operation log, determine the columns not involved in all the new column value information of the current operation log, and obtain the old column value information to be inserted of the determined columns; based on all the new column value information of the current operation log and the old column value information to be inserted, generate the target fusion operation.

[0133] For the case "INS+UPD", in an optional embodiment, as Fig.13As shown, all the columns involved in the old column value information OP_OLD_VAL of the current operation log are traversed. When one of the columns COLi is not involved in all the new column value information OP_NEW_VAL of the current operation log, the old column value information of the column COLi is obtained. Finally, according to all the new column value information OP_NEW_VAL and the old column value information of the column COLi, the column value information required to generate the target fusion operation is obtained, and then the insert operation is generated. Fig.14 As shown, all the old column value information OP_OLD_VAL of the current operation log is (c1=2, c2=1), and all the new column value information OP_NEW_VAL of the current operation log is (c2=2), wherein column c1 is an uninvolved column, so "c1=2" is obtained from all the old column value information of the current operation log; finally, "c1=2" and "c2=2" are used as column values ​​of the insert operation to generate the insert operation.

[0134] Step 30422: When the operation type of the first historical operation log is an insert operation, and the operation type of the current operation log is a delete operation, a target fusion operation is not generated, so that the target fusion operation is not subsequently executed in the target database.

[0135] For the case of "INS+DEL", since an insert operation is first performed on the same data row and then a delete operation is performed, it is equivalent to not directly performing an insert operation on the data row. Therefore, the embodiment of the present invention ignores the current operation log, does not generate a target fusion operation, and keeps the data row in the source database and the target database consistent.

[0136] Step 30423: When the operation type of the first historical operation log is an update operation, and the operation type of the current operation log is an update operation, the operation type of the target fusion operation is an update operation; based on all the old column value information of the first historical operation log and the current operation log, determine the column value information before the update and the column value information after the update to generate the target fusion operation.

[0137] For the case of "UPD+UPD", all the old column value information of the first historical operation log is used as the column value information before the update. Fig.15 and Fig.16 As shown, all old column value information OP_OLD_VAL of the first historical operation log, that is, (C1=1, C2=1), is obtained as the pre-update column value information OP_OLD_VAL of the target fusion operation.

[0138] From the columns involved in the column value information before the update, determine the columns not involved in all the new column value information of the current operation log. For example, from (C1=1, C2=1), determine the columns not involved in all the new column value information OP_NEW_VAL of the current operation log, that is, column C1; from all the old column value information OP_OLD_VAL of the current operation log, obtain the column value information to be updated of column C1 as "C1=3".

[0139] From all the old column value information of the current operation log, the column value information to be updated of the determined column is obtained; based on all the new column value information of the current operation log and the column value information to be updated, the updated column value information is obtained. For example, based on all the new column value information OP_NEW_VAL of the current operation log and the column value information to be updated "C1=3" of column C1, the updated column value information OP_NEW_VAL of the target fusion operation is obtained.

[0140] The target fusion operation is generated based on the pre-update column value information and the post-update column value information. For example, "c1=1" and "c2=1" are used as the pre-update column value information of the update operation, and "c1=3" and "c2=2" are used as the post-update column value information of the update operation to generate the update operation.

[0141] Step 30424: When the operation type of the first historical operation log is an update operation, and the operation type of the current operation log is a delete operation, the operation type of the target fusion operation is a delete operation; all old column value information of the first historical operation log is determined as the old column value information to be deleted, and the target fusion operation is generated based on the old column value information to be deleted.

[0142] For the case "UPD+DEL", such as Fig.17 As shown, all old column value information of the first historical operation log is (c1=1, c2=1), and "c1=1" and "c2=1" are used as column values ​​of the deletion operation to generate the deletion operation.

[0143] When all operation logs corresponding to the pending transaction are processed, all operation logs corresponding to the pending transaction are deleted from the operation registration linked list. Fig.18As shown, when synchronizing, the synchronization tool first obtains TRX_INFO from TRX_LST in sequence. If TRX_INFO has a record in TRX_TABLE, the TRX_INFO is ignored; otherwise, the operation log OP is obtained from TRX_INFO in sequence. When the operation log OP has a conflict association relationship (that is, OP.op_next is not empty), if the operation log OP is not the tail of the data conflict operation, the operation log OP is ignored and the next operation log OP is obtained; when OP.op_base is not empty, the conflict operation fusion method is used to obtain the target fusion operation, and synchronization is performed in the target database. When all the operation logs in TRX_INFO are executed, the operation log belonging to the TRX_INFO is deleted from OPS_LST, and the TRX_INFO information is registered in the submitted transaction registration table TRX_TABLE.

[0144] When the synchronization tool crashes and restarts during the synchronization process, an embodiment of the present invention provides a method for ensuring data consistency: during the operation of the synchronization tool, each time at intervals of the second preset time, the source end SCN of the latest operation log that has completed synchronization is used to update the starting SCN; in the operation registration linked list, all operation logs before the starting SCN of the corresponding source end SCN are determined, and the conflicting operation pointer of the determined operation log is saved to a file, and the operation log pointed to by the conflicting operation pointer (i.e., the corresponding conflicting operation header) is saved to a file. Among them, the second preset time is selected by those skilled in the art based on the specific usage scenario and experience. The starting SCN is the source end SCN of the first operation log to be synchronized after the synchronization tool is restarted. For example, in OP_LST, the operation logs of the source end SCN before the starting SCN are operation log OP1, operation log OPi, and operation log OPn. Since there is no conflicting operation pointer in operation log OP1, the conflicting operation pointer op_base of operation log OPi and the conflicting operation pointer op_base of operation log OPn are written to the file for storage, and the operation log OP1 is saved to the file.

[0145] On this basis, each time the synchronization between the source database and the destination database is restarted, the conflicting operation pointers and operation logs saved in the file are loaded into the memory, so that the corresponding conflicting operation pointers are restored to the operation registration list; for example, the conflicting operation pointers op_base of the operation log OPi, the conflicting operation pointers op_base of the operation log OPn, and the operation log OP1 are loaded from the file into the memory and restored to the operation registration list OPS_LST, so that OP_LST is restored to the state at the start SCN time. After OP_LST is restored, the synchronization tool starts from the start SCN and resynchronizes according to the method for accelerating data synchronization of the embodiment of the present invention.

[0146] Embodiment 2:

[0147] The embodiment of the present invention is a preferred embodiment of embodiment 1, such as Fig.19 As shown, it also includes:

[0148] In step 501, if the operation type of the current operation log is a delete operation, the target fusion operation is selectively not generated, or the target fusion operation is immediately generated based on the current operation log and the corresponding first historical operation log, and the target fusion operation is executed in the target database.

[0149] For a Fig. 9 The conflicting association sequence shown in the figure, according to the description in Example 1, the operation type at the end of the conflicting operation is a delete operation, which includes two cases: (1) Case "INS+DEL", which is equivalent to not performing an insert operation on the data row, omitting all operations in the conflicting association sequence. Therefore, in this case, the target fusion operation is not generated directly, that is, the synchronization of the conflicting association sequence is completed. (2) Case "UPD+DEL", which is equivalent to a partial update operation on the data row that will be deleted later. In order to speed up the synchronization, when the current operation log is processed, a delete operation (i.e., the target fusion operation) is immediately generated and executed, omitting some update operations that will be deleted later.

[0150] In step 502, if the operation type of the current operation log is not a deletion operation, the corresponding target fusion operation is not generated temporarily, and it is determined whether the conflict association relationship sequence corresponding to the current operation log increases within a first preset time.

[0151] The first preset time is selected by technicians in this field according to the specific usage scenario and experience.

[0152] In step 503, when the corresponding conflict association relationship sequence is not increased within the first preset time, a target fusion operation is generated according to the current operation log and the corresponding first historical operation log, and the target fusion operation is executed in the target database.

[0153] In step 504, when the corresponding conflict association relationship sequence increases within the first preset time, a corresponding target fusion operation is generated according to the last operation log in the increased conflict association relationship sequence and the corresponding first historical operation log, and the target fusion operation is executed in the target database.

[0154] In actual application scenarios, the synchronization tool may process multiple operation logs for the same data row within a period of time. If the target fusion operation is generated and executed based on the conflict association sequence when processing the first operation log, then when the next operation log for the same data row is read later, the conflict association sequence needs to be re-established, which will affect the synchronization speed. For example, in the source database, operation log OP1, operation log OPi, operation log OPj, operation log OPn1, operation log OPn2 and operation log OPn3 are operation logs for the same data row. When the synchronization tool only registers operation log OP1, operation log OPi, operation log OPj and operation log OPn1 to the operation registration linked list, operation log OP1, operation log OPi, operation log OPj and operation log OPn1 constitute the first conflict association sequence. The tool first ignores the operation log OP1, operation log OPi and operation log OPj in the processing order of the transactions to be executed. When processing the operation log OPn1, based on the first conflict association relationship sequence, the first target fusion operation is generated and executed according to the operation log OP1 and the operation log OPn1 to complete the synchronization of the operation log OP1, the operation log OPi, the operation log OPj and the operation log OPn1; when the first target fusion operation is generated and executed, the operation log OPn2 and the operation log OPn3 are registered to the operation registration linked list immediately, then it is necessary to re-establish the second conflict association relationship sequence, and ignore the operation log OPn2 in the processing order of the transactions to be executed. When processing the operation log OPn3, based on the second conflict association relationship sequence, the second target fusion operation is generated and executed according to the operation log OPn2 and the operation log OPn3 to complete the synchronization of the operation log OPn2 and the operation log OPn3.

[0155] In view of this situation, when the operation type of the conflict operation tail of the conflict association sequence is not a deletion operation, the embodiment of the present invention performs a second preset time wait for the subsequent operation log; if the conflict operation tail of the conflict association sequence changes during the second preset time (i.e., the conflict operation tail is a newly added operation log), the target fusion operation is generated according to the newly added operation log to further speed up the synchronization speed by ignoring more operation logs. It should be noted that since the deletion operation has more execution steps that can be omitted and is most effective in speeding up the synchronization speed, it has the highest execution priority, and only when the conflict operation tail is not a deletion operation, the waiting is performed.

[0156] Embodiment 3:

[0157] like Fig. 20, which is a schematic diagram of the architecture of a device for accelerating data synchronization according to an embodiment of the present invention. The device for accelerating data synchronization according to this embodiment includes one or more processors 21 and a memory 22. Fig. 20 A processor 21 is taken as an example.

[0158] The processor 21 and the memory 22 may be connected via a bus or other means. Fig. 20 The example of connecting through bus is taken in the following.

[0159] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the method for accelerating data synchronization in this embodiment. The processor 21 executes the method for accelerating data synchronization by running the non-volatile software programs and instructions stored in the memory 22.

[0160] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the method for accelerating data synchronization in the above embodiment is executed, for example, the method described above is executed. Figure 1 , Figure 3 , Figure 5 , Figure 8 , Fig.12 and Fig.19 The steps shown.

[0162] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Fig. 20 A processor 21 can enable the one or more processors to execute the method for accelerating data synchronization in a specific embodiment of the present invention, for example, executing the above-described Figure 1 , Figure 3 , Figure 5 , Figure 8 , Fig.12 and Fig.19 The steps shown can also be implemented Fig. 20 The modules and units described above; or executing the method for accelerating data synchronization in a specific embodiment of the present invention, for example, executing the method described above Figure 1, Figure 3 , Figure 5 , Figure 8 , Fig.12 and Fig.19 The steps shown can also be implemented Fig. 20 The various modules and units described.

[0163] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0164] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for accelerating data synchronization, characterized in that: include: Obtain the current operation log from the source database, and perform a conflict check between the current operation log and all historical operation logs in the operation registration linked list; When at least one historical operation log conflicts with the current operation log, the current operation log is added to the operation registration linked list to establish a conflict association relationship between the at least one historical operation log and the current operation log; Generate a corresponding target fusion operation according to the conflict association relationship, the at least one historical operation log and the current operation log; The target fusion operation is performed in the target database to complete data synchronization in the target database according to the at least one historical operation log and the current operation log.

2. The method for accelerating data synchronization according to claim 1, characterized in that: The obtaining of the current operation log from the source database and the conflict check between the current operation log and all the historical operation logs in the operation registration linked list include: Obtain the current operation logs from the source database in ascending order of the source SCN; For the current operation log, obtain the LSCN and ROWID of the current operation log; wherein the LSCN of the operation log is the time when the data operated by the operation log was last modified, and the ROWID of the operation log is the ROWID of the data operated by the operation log; According to the order in which the historical operation logs are added to the operation registration linked list, the LSCN and ROWID of the historical operation logs are sequentially obtained from the operation registration linked list; If the LSCN of the current operation log is inconsistent with the LSCN of the historical operation log, there is no conflict between the current operation log and the historical operation log; if the LSCN of the current operation log is consistent with the LSCN of the historical operation log, it is determined whether the ROWID of the current operation log is consistent with the ROWID of the historical operation log; When the ROWID of the current operation log is consistent with the ROWID of the historical operation log, there is a conflict between the current operation log and the historical operation log, so as to complete the conflict check between the current operation log and the historical operation log.

3. The method for accelerating data synchronization according to claim 1, characterized in that: When there is a conflict between at least one historical operation log and the current operation log, adding the current operation log to the operation registration linked list, and establishing a conflict association relationship between the at least one historical operation log and the current operation log comprises: When the current operation log is added to the operation registration linked list, a conflict association relationship is established between each historical operation log and its next historical operation log in pairs according to the order in which the at least one historical operation log is added to the operation registration linked list; A conflict association relationship is established between the last historical operation log in the at least one historical operation log and the current operation log.

4. The method for accelerating data synchronization according to claim 1, characterized in that: The generating a corresponding target fusion operation according to the conflict association relationship, the at least one historical operation log and the current operation log includes: Obtaining the operation log corresponding to the transaction to be executed in the operation registration linked list; When the acquired operation logs have conflicting association relationships, acquiring a conflicting association relationship sequence associated with the acquired operation logs; wherein a conflicting association relationship sequence includes at least one conflicting association relationship between the operation logs; When the conflict association relationship of the obtained operation log is not the last conflict association relationship in the conflict association relationship sequence, ignoring the obtained operation log, and obtaining the next operation log of the obtained operation log in the to-be-executed transaction; When the conflict association relationship of the acquired operation log is the last conflict association relationship in the conflict association relationship sequence, a target fusion operation corresponding to the conflict association relationship sequence is generated according to the operation logs in the conflict association relationship sequence.

5. The method for accelerating data synchronization according to claim 4, characterized in that: When the conflict association relationship of the acquired operation log is the last conflict association relationship in the conflict association relationship sequence, generating a target fusion operation corresponding to the conflict association relationship sequence according to the operation log in the conflict association relationship sequence includes: Acquire the first historical operation log in the conflict association relationship sequence according to the conflict operation pointer; except for the first historical operation log, each operation log corresponding to the conflict association relationship sequence has a conflict operation pointer, and the corresponding conflict operation pointers all point to the first historical operation log; The target fusion operation is generated according to the operation type and column value information of the first historical operation log and the current historical operation log.

6. The method for accelerating data synchronization according to claim 5, characterized in that: The generating the target fusion operation according to the operation type and column value information of the first historical operation log and the current historical operation log includes: When the operation type of the first historical operation log is an insert operation, and the operation type of the current operation log is an update operation, the operation type of the target fusion operation is an insert operation; from the columns involved in all the old column value information of the current operation log, determine the columns not involved in all the new column value information of the current operation log, and obtain the old column value information to be inserted of the determined columns; based on all the new column value information of the current operation log and the old column value information to be inserted, generate the target fusion operation; When the operation type of the first historical operation log is an insert operation, and the operation type of the current operation log is a delete operation, a target fusion operation is not generated, so that the target fusion operation is not subsequently executed in the target database; When the operation type of the first historical operation log is an update operation, and the operation type of the current operation log is an update operation, the operation type of the target fusion operation is an update operation; according to all the old column value information of the first historical operation log and the current operation log, the column value information before the update and the column value information after the update are determined to generate the target fusion operation; When the operation type of the first historical operation log is an update operation, and the operation type of the current operation log is a delete operation, the operation type of the target fusion operation is a delete operation; all old column value information of the first historical operation log is determined as the old column value information to be deleted, and the target fusion operation is generated according to the old column value information to be deleted.

7. The method for accelerating data synchronization according to claim 6, characterized in that: The determining, based on all old column value information of the first historical operation log and the current operation log, the column value information before and after the update to generate the target fusion operation includes: Using all old column value information of the first historical operation log as column value information before updating; Determine, from the columns involved in the column value information before the update, the columns not involved in all the new column value information of the current operation log; obtain, from all the old column value information of the current operation log, the column value information to be updated of the determined columns; Based on all new column value information of the current operation log and the column value information to be updated, obtaining updated column value information; The target fusion operation is generated based on the pre-update column value information and the post-update column value information.

8. The method for accelerating data synchronization according to any one of claims 1 to 7, characterized in that: Also includes: If the operation type of the current operation log is a delete operation, then selectively not generating a target fusion operation, or immediately generating a target fusion operation according to the current operation log and the corresponding first historical operation log, and executing the target fusion operation in the destination database; If the operation type of the current operation log is not a delete operation, the corresponding target fusion operation is not generated temporarily, and it is determined whether the conflict association relationship sequence corresponding to the current operation log increases within the first preset time; When the corresponding conflict association relationship sequence is not increased within the first preset time, generating a target fusion operation according to the current operation log and the corresponding first historical operation log, and executing the target fusion operation in the target database; When the corresponding conflict association relationship sequence increases within the first preset time, a corresponding target fusion operation is generated according to the last operation log in the increased conflict association relationship sequence and the corresponding first historical operation log, and the target fusion operation is executed in the target database.

9. A device for accelerating data synchronization, characterized in that: The device for accelerating data synchronization includes at least one processor and a memory, and the at least one processor and the memory are connected via a data bus. The memory stores instructions that can be executed by the at least one processor. After being executed by the processor, the instructions are used to implement the method for accelerating data synchronization described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method for accelerating data synchronization described in any one of claims 1-8.

Images