Data synchronization method and device and storage medium

Abstract

The invention discloses a data synchronization method and device and a storage medium, and belongs to the field of communication. The method comprises the steps that multiple first operation records are obtained, the multiple first operation records correspond to multiple SQL statements executed by a source database system on a source database, and each first operation record is used for describing the SQL statement corresponding to the corresponding first operation record; the SQL statement corresponding to each first operation record is a statement used for processing the first record; a target SQL statement is obtained based on the multiple first operation records, and a result generated by processing the first records through the multiple SQL statements is the same as a result generated by processing the first records through the target SQL statement; and sending the target SQL statement to a target database system, wherein the target database system is used for executing the target SQL statement on the target database to complete data synchronization. According to the invention, data synchronization and efficiency can be improved.

Description

[0001] This application claims priority to Chinese patent application No. 202311132402.2, filed on September 4, 2023, and entitled “A data synchronization method, apparatus and computing device cluster”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communications, and in particular to a method, device and storage medium for data synchronization. Background Art

[0003] The database system is a system composed of a database and its management software. The database is a warehouse for storing data. The database system includes a source database system and a target database system. The target database included in the target database system is used to synchronize the source database included in the source database system.

[0004] The source database system may receive a first structured query language (SQL) statement for the source database, process the source database by executing the first SQL statement, and save an operation record in a log, where the operation record is used to describe the first SQL statement. The source database system may send the log to a data synchronization tool, which, after receiving the log, restores the operation record into a second SQL statement and sends the second SQL statement to a target database system, where the function implemented by the first SQL statement is the same as the function implemented by the second SQL statement. The target database system processes the target database by executing the second SQL statement to achieve data synchronization.

[0005] In the scenario of storing extremely large amounts of data, the log includes a large number of operation records. A large number of SQL statements will be restored based on the large number of operation records. It takes a long time to process the target database using a large number of SQL statements, which reduces the efficiency of data synchronization. Summary of the invention

[0006] The present application provides a method, device and storage medium for data synchronization to improve the processing efficiency of a database. The technical solution is as follows:

[0007] In a first aspect, the present application provides a method for data synchronization, the method being performed by a database synchronization tool, the database synchronization tool being used to synchronize data in a source database to a target database. In the method, a plurality of first operation records are obtained, the plurality of first operation records corresponding to a plurality of structured query language SQL statements executed by a source database system on a source database, each first operation record being used to describe a SQL statement corresponding to each first operation record, and the SQL statement corresponding to each first operation record is a statement for processing the first record. A target SQL statement is obtained based on the plurality of first operation records, wherein the result generated by processing the first record using the plurality of SQL statements is the same as the result generated by processing the first record using the target SQL statement. The target SQL statement is sent to a target database system, the target database system being used to execute the target SQL statement on the target database to complete data synchronization.

[0008] A plurality of first operation records are obtained. Since the plurality of SQL statements corresponding to the plurality of first operation records are used to process the plurality of SQL statements of the first record, a target SQL statement is obtained based on the plurality of first operation records. The result generated by processing the first record using the plurality of SQL statements is the same as the result generated by processing the first record using the target SQL statement. This reduces the number of SQL statements sent to the target database system and the number of SQL statements executed by the target database system on the target database, thereby improving the efficiency of data synchronization.

[0009] In a possible implementation, the first operation record includes a first operation type and first data belonging to the first record, and the first operation type is an operation type of the first record processed by a SQL statement corresponding to the first operation record.

[0010] In another possible implementation, based on the first operation type included in each first operation record and the execution order of multiple SQL statements corresponding to the multiple first operation records, the operation type included in the target SQL statement is obtained. Based on the first data included in each first operation record, the data belonging to the first record included in the target SQL statement is obtained. In this way, the multiple SQL statements corresponding to the multiple first operation records are merged into the target SQL statement, reducing the number of SQL statements sent to the target database system.

[0011] In another possible implementation, the second operation record and the third operation record are obtained, the second operation record includes the second operation type and the second data belonging to the second record, and the third operation record includes the third operation type and the third data belonging to the second record. When the second operation type is inserting the second record and the third operation type is deleting the second record, the second operation record and the third operation record are discarded. In this way, the number of operation records that need to be processed can be reduced, thereby reducing the number of target SQL statements obtained and improving the efficiency of data synchronization.

[0012] In another possible implementation, the first operation record also includes identification information of a data table to which the first record belongs, the first data belonging to the first record includes primary key information of the first record, and the data table is a data table in a source database. A log of the source database is obtained, and the log includes multiple operation records. Multiple operation records including the same primary key information and identification information of the same data table are selected from the log as multiple first operation records. In this way, multiple operation records for processing the same record can be accurately obtained according to the primary key information, simplifying the implementation complexity of obtaining multiple first operation records.

[0013] In another possible implementation, the multiple operation records include a fourth operation record, and the SQL statement described in the fourth operation record is used to update the first primary key information of the first record to the second primary key information. The fourth operation record included in the log is replaced with a fifth operation record and a sixth operation record, the operation type in the fifth operation record is to delete the first record, the data belonging to the first record in the fifth operation record includes the first primary key information, the operation type in the sixth operation record is to insert the first record, the data belonging to the first record in the sixth operation record includes the second primary key information and the data in the first record except the first primary key information, and the fifth operation record and the sixth operation record correspond to the SQL statement described in the fourth operation record. In this way, it is convenient to obtain multiple first operation records for processing the same record according to the primary key information, and the implementation complexity of obtaining multiple first operation records is simplified.

[0014] In a second aspect, the present application provides a data synchronization device, which is used to execute the method in the first aspect or any possible implementation of the first aspect. Specifically, the device includes a unit for executing the method in the first aspect or any possible implementation of the first aspect.

[0015] In a third aspect, the present application provides a computing device cluster, the computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory;

[0016] The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method in the first aspect or any possible implementation manner of the first aspect.

[0017] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, causes the computing device cluster to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0018] In a fifth aspect, the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method in the first aspect or any possible implementation of the first aspect.

[0019] In a sixth aspect, the present application provides a chip comprising a memory and a processor, the memory being used to store computer instructions, and the processor being used to call and run the computer instructions from the memory to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the structure of a database system provided in an embodiment of the present application;

[0021] Figure 2 It is a structural diagram of a network architecture provided by an embodiment of the present application;

[0022] Figure 3 It is a structural diagram of another network architecture provided by an embodiment of the present application;

[0023] Figure 4 This is a flow chart of a data synchronization method provided by an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the structure of a data synchronization device provided in an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0026] Figure 7 This is a schematic diagram of a cluster structure for data synchronization provided by an embodiment of the present application;

[0027] Figure 8 This is another schematic diagram of a cluster structure for data synchronization provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] See also Figure 1An embodiment of the present application provides a database system 100 , where the database system 100 includes a database 101 , and the database system 100 can process the database 101 .

[0029] For example, the processing operation of the database system 100 on the database 101 may include one or more of the following processing operations: inserting data into the database 101 , updating data stored in the database 101 , or deleting data stored in the database 101 .

[0030] In some embodiments, the database system 100 may process the database 101 by executing an SQL statement. Optionally, during implementation: the database system 100 may receive an SQL statement and process the database 101 by executing the SQL statement.

[0031] For example, the database system 100 may receive an insert statement and insert data into the database 101 based on the insert statement. For another example, the database system 100 may receive an update statement and update the data stored in the database 101 based on the update statement. For another example, the database system 100 may receive a delete statement and delete the data stored in the database 101 based on the delete statement.

[0032] The data stored in the database 101 includes one or more rows of records. The database system 100 processes the database 101 , which means that the database system 100 processes the records belonging to the database 101 .

[0033] For example, the database system 100 may receive an insert statement and insert a row of records into the database 101 based on the insert statement. For another example, the database system 100 may receive an update statement and update the data in one or more rows of records stored in the database 101 based on the update statement. For another example, the database system 100 may receive a delete statement and delete one or more rows of records stored in the database 101 based on the delete statement.

[0034] The database 101 includes one or more data tables, each of which stores one or more rows of records, and the records belonging to the database 101 are records belonging to the data tables included in the database 101. The database system 100 processes the records belonging to the database 101, which may be: the database system 100 processes the records belonging to the data tables included in the database 101.

[0035] After receiving at least one SQL statement, the database system 100 allocates a transaction to the at least one SQL statement and connects to the database 101. Then, in the transaction, the records belonging to the database 101 are processed based on the at least one SQL statement, that is, the at least one SQL statement is executed in the transaction.

[0036] In some embodiments, the database system 100 further includes a database driver 102 . The database system 100 processes the database 101 , which may be: the database driver 102 processes the database 101 .

[0037] That is, the database driver 102 can process the database 101 by executing SQL statements. Optionally, during implementation: the database driver 102 can receive an SQL statement and process the database 101 by executing the SQL statement.

[0038] For example, the database driver 102 may receive an insert statement and insert data into the database 101 based on the insert statement. For another example, the database driver 102 may receive an update statement and update the data stored in the database 101 based on the update statement. For another example, the database driver 102 may receive a delete statement and delete the data stored in the database 101 based on the delete statement.

[0039] The database included in a single database system may not be able to be restored when data is lost. To improve data security, see Figure 2 The network architecture 20 shown includes a source database system 100a, a database synchronization tool 200, and a target database system 100b. The source database system 100a includes a source database 101a, and the target database system 100b includes a target database 101b.

[0040] The database synchronization tool 200 is used to synchronize data in the source database 101a to the target database 101b, so that the target database 101b can back up the source database 101a. In this way, when data is lost in the source database 101a, the lost data can be restored through the target database 101b.

[0041] The source database system 100a can receive multiple SQL statements 1 for the source database 101a, process the source database 101a by executing the multiple SQL statements 1, and save multiple operation records in the log of the source database 101a, the multiple operation records correspond to the multiple SQL statements 1, and each operation record is used to describe the SQL statement 1 corresponding to each operation record.

[0042] The database synchronization tool 200 can obtain the log of the source database 101a, and based on the log, obtain multiple SQL statements 2 corresponding to the multiple SQL statements 1, and the function implemented by each SQL statement 1 is respectively the same as the function implemented by the SQL statement 2 corresponding to each SQL statement 1. The database synchronization tool 200 sends the multiple SQL statements 2 to the target database system 100b. The target database system 100b processes the target database 101b by executing the multiple SQL statements 2 to synchronize the source database 101a with the target database 101b.

[0043] In some embodiments, the source database system 100a further includes a source database driver 102a, and the target database system 100b further includes a target database driver 102b.

[0044] Optionally, the operation of the source database system 100a processing the source database 101a may be: the source database driver 102a receives the multiple SQL statements 1, processes the source database 101a by executing the multiple SQL statements 1, and saves the multiple operation records in the log of the source database 101a.

[0045] Optionally, the operation of the target database system 100b processing the target database 101b by executing the multiple SQL statements 2 may be: the target database driver 102b receives the multiple SQL statements 2, and processes the target database 101b by executing the multiple SQL statements 2 to synchronize the source database 101a and the target database 101b.

[0046] After receiving the multiple SQL statements 2, the target database system 100b executes each SQL statement 2 to process the records belonging to the target database 101b using the SQL statement 2. Therefore, the target database system 100b needs to spend some time to execute one SQL statement 2.

[0047] If the target database system 100b receives a large number of SQL statements 2, it will take a while for the target database system 100b to execute each SQL statement 2, so the target database system 100b needs a lot of time to execute the large number of SQL statements 2, which wastes a lot of time, reduces the efficiency of processing the target database 101b, and further reduces the efficiency of data synchronization.

[0048] If the number of SQL statements received by the target database system 100b can be reduced, the time required for the target database system 100b to execute SQL statements can undoubtedly be reduced, thereby improving the efficiency of the target database system 100b in processing the target database 101b, that is, improving the efficiency of data synchronization.

[0049] For this purpose, see Figure 3 After obtaining the log of the source database 101a, the database synchronization tool 200 obtains multiple first operation records based on the log, and the multiple first operation records correspond to multiple SQL statements 1 executed by the source database system 100a on the source database 101a. The multiple SQL statements 1 are statements for processing first records, and the first records belong to the source database 101a. The target SQL statement is obtained based on the multiple first operation records, wherein the result of processing the first record using the multiple SQL statements 1 is the same as the result of processing the first record using the target SQL statement. The target SQL statement is sent to the target database system 100b. In this way, the target database system 100b executes the target SQL statement on the target database 101b to complete data synchronization, thereby reducing the number of SQL statements executed by the target database system 100b and improving data synchronization efficiency.

[0050] See also Figure 4 The present application embodiment provides a data synchronization method 400, the method 400 is applied to Figure 3 In the network architecture 20 shown, the method 400 includes the following process.

[0051] Step 401: a source database system receives M SQL statements, where the M SQL statements are statements that need to be executed by the source database system, the source database system includes a source database, and M is an integer greater than 1.

[0052] The M SQL statements are sent by a user. The user sends the M SQL statements to the source database system, and the source database system receives the M SQL statements.

[0053] For each of the M SQL statements, the SQL statement is used to process a record belonging to a source database, which is a database located in a source database system. The record may be a record to be inserted into the source database, or the record is a record stored in the source database. The SQL statement includes an operation type, data belonging to the record, and identification information of a data table to which the record belongs, the data belonging to the record includes primary key information of the record, and the data table is a data table in the source database.

[0054] Optionally, the data belonging to the record may also include data other than the primary key information.

[0055] For example, suppose the SQL statement is the following insert statement:

[0056] insert into information(ID, name, age, career)

[0057] value(1,A,18,student);

[0058] This SQL statement is used to insert a record into the user information table. The SQL statement includes the operation type insert, the identification information of the user information table is information and the record. The data included in the record is a user information, and the record includes the primary key information ID=1, the user name name=A, the user age age=18, and the user occupation career=student. The operation type insert is used to indicate inserting the record into the user information table.

[0059] Assume that the SQL statement is an update statement as follows:

[0060] update information

[0061] set age=19

[0062] where ID = 1;

[0063] This SQL statement is used to update a record stored in the user information table. The SQL statement includes the operation type of update, the identification information of the user information table is information, and the data belonging to the record. The data belonging to the record includes primary key information ID=1, and the updated user age age=19. For the record stored in the user information table and including primary key information ID=1, the operation type update is used to indicate that the user age age stored in the record is updated from 18 to 19.

[0064] Assume that the SQL statement is the following delete statement:

[0065] delete information

[0066] where ID = 1;

[0067] This SQL statement is used to delete a record stored in the user information table. The SQL statement includes an operation type of delete, identification information of the user information table of information and data belonging to the record. The data belonging to the record includes primary key information ID=1. For the record stored in the user information table including primary key information ID=1, the operation type delete is used to indicate deleting the record.

[0068] Step 402: The source database system executes the M SQL statements on the source database to process the source database, and saves M operation records to the log of the source database, where the M operation records are used to describe the M SQL statements.

[0069] For each SQL statement, the source database driver included in the source database system executes the SQL statement on the source database to process the records belonging to the source database, and saves an operation record describing the SQL statement to the log of the source database. The operation record includes the operation type in the SQL statement, the data belonging to the record, and the identification information of the data table to which the record belongs. The data belonging to the record includes the primary key information of the record, so the operation record includes the operation type in the SQL statement, the primary key information of the record, and the identification information of the data table to which the record belongs.

[0070] For example, assume that the M SQL statements include the following three SQL statements, which are SQL statement A, SQL statement B, and SQL statement C.

[0071] Among them, SQL statement A is:

[0072] insert into information(ID, name, age, career)

[0073] value(1,A,18,student);

[0074] SQL statement B is:

[0075] update information

[0076] set age=19

[0077] where ID = 1;

[0078] And, SQL statement C is:

[0079] update information

[0080] set career=cashier

[0081] where ID=1.

[0082] SQL statement A is used to insert a record into the user information table. The record is ID=1, name=A, age=18, career=student, and information is the identification information of the user information table. SQL statement B is used to update the age in the record from 18 to 19. SQL statement C is used to update the career in the record from student to cashier.

[0083] For SQL statement A, the source database driver included in the source database system executes SQL statement A on the source database to insert the record in the user information table included in the source database, as shown in Table 1 below, where the record is ID=1, name=A, age=18, and career=student. The source database driver saves operation record 1 to the log of the source database, and operation record 1 includes the operation type insert in SQL statement A, the data belonging to the record: ID=1, name=A, age=18, career=student, and the identification information information of the user information table. Operation record 1 can be expressed as: insert; data: ID=1, name=A, age=18, career=student; information.

[0084] Table 1

[0085] ID name age career 1 A 18 student

[0086] For SQL statement B, the source database driver included in the source database system executes SQL statement B on the source database shown in Table 1, and updates age in the record shown in Table 1 from 18 to 19. The result is shown in Table 2 below. The source database driver saves operation record 2 to the log of the source database. Operation record 2 includes the operation type update in SQL statement B, the data belonging to the record: ID=1, age=19, and the identification information of the user information table information. Operation record 2 can be expressed as: update; data: ID=1, age=19; information.

[0087] Table 2

[0088] ID name age career 1 A 19 student

[0089] For SQL statement C, the source database driver included in the source database system executes SQL statement C on the source database shown in Table 2, and updates the career in the record shown in Table 2 from student to cashier, and the result is shown in Table 3 below. The source database driver saves operation record 3 to the log of the source database, and operation record 3 includes the operation type update in SQL statement C, the data belonging to the record: ID = 1, career = cashier, and the identification information information of the user information table. Operation record 3 can be expressed as: update; data: ID = 1, career = cashier; information.

[0090] Table 3

[0091] ID name age career 1 A 19 cashier

[0092] In some embodiments, if the M SQL statements include an update statement, the update statement is used to update the first primary key information in the record to the second primary key information. The source database driver included in the source database system executes the update statement on the source database, updates the first primary key information in the record included in the source database to the second primary key information, and saves an operation record describing the update statement to the log of the source database. The operation record includes the operation type update in the update statement, the data belonging to the record, and the identification information of the data table to which the record belongs, and the data belonging to the record includes the first primary key information and the second primary key information.

[0093] For example, it is assumed that the M SQL statements include, in addition to the above-mentioned SQL statement A, SQL statement B, and SQL statement C, also include the SQL statement D shown below.

[0094] update information

[0095] set ID = 2;

[0096] SQL statement D is used to update the first primary key information ID=1 in the record shown in Table 3 to the second primary key information ID=2.

[0097] The source database driver included in the source database system executes the update statement on the source database shown in Table 3, and updates the first primary key information (ID=1) in the record included in the source database shown in Table 3 to the second primary key information (ID=2), and the result is shown in Table 4 below. The source database driver saves the operation record 4 describing the update statement to the log of the source database. The operation record 4 includes the operation type update in the update statement, the data belonging to the record: ID=1, ID=2, and the identification information information of the user information table. The operation record 4 can be expressed as: update; data: ID=1, ID=2; information.

[0098] Table 4

[0099] ID name age career 2 A 19 cashier

[0100] After the source database system executes the M SQL statements on the source database, the source database system may also receive other SQL statements, execute the other SQL statements on the source database, and save operation records describing the other SQL statements to the log of the source database.

[0101] After the source database system executes SQL statements on the source database, the database synchronization tool can synchronize the data in the source database to the target database. The detailed implementation process is as follows.

[0102] Step 403: The database synchronization tool obtains a log of the source database, where the log includes multiple operation records, and the multiple operation records are used to describe multiple SQL statements executed by the source database system on the source database.

[0103] In some embodiments, the source database system may send the log of the source database to the database synchronization tool, and the database synchronization tool receives the log of the source database.

[0104] For example, the log of the source database saved by the source database system includes the above-mentioned operation record 1, operation record 2, and operation record 3, or the log of the source database saved by the source database system includes the above-mentioned operation record 1, operation record 2, operation record 3, and operation record 4. The source database system sends the log to the database synchronization tool, and the database synchronization tool receives the log.

[0105] Step 404: the database synchronization tool obtains multiple first operation records from the log of the source database, the multiple first operation records correspond to multiple SQL statements executed by the source database system on the source database, the multiple SQL statements are statements for processing the first records, and the first records are records belonging to the source database.

[0106] For each first operation record, the first operation record includes a first operation type, first data belonging to the first record, and identification information of a data table to which the first record belongs, and the data belonging to the first record includes primary key information of the first record.

[0107] In step 404, the database synchronization tool selects a plurality of operation records including the same primary key information and identification information of the same data table from the log of the source database as a plurality of first operation records.

[0108] For example, the log of the source database includes the above-mentioned operation record 1, operation record 2, and operation record 3. Operation record 1 is represented as: insert; data: ID=1, name=A, age=18, career=student; information. Operation record 2 is represented as: update; data: ID=1, age=19; information. Operation record 3 is represented as: update; data: ID=1, career=cashier; information. The database synchronization tool selects operation record 1, operation record 2, and operation record 3 that include the same primary key information (ID=1) and identification information (information) of the same data table from the log, and obtains three first operation records.

[0109] In some embodiments, the log of the source database may include a second operation record and a third operation record, the second operation record includes a second operation type and second data belonging to the second record, and the third operation record includes a third operation type and third data belonging to the second record. In step 404, the database synchronization tool obtains the second operation record and the third operation record including the same primary key information and identification information of the same data table from the log of the source database, and the primary key information is the primary key information of the second record; when the second operation type is inserting the second record and the third operation type is deleting the second record, the second operation record and the third operation record are discarded.

[0110] In some embodiments, the log of the source database may include a fourth operation record, and the SQL statement described in the fourth operation record is an update statement for updating the first primary key information of the first record to the second primary key information. Therefore, before the database synchronization tool selects multiple operation records including the same primary key information and identification information of the same data table from the log of the source database, it replaces the fourth operation record included in the log of the source database with the fifth operation record and the sixth operation record, the operation type in the fifth operation record is to delete the first record, the data belonging to the first record in the fifth operation record includes the first primary key information, the operation type in the sixth operation record is to insert the first record, the data belonging to the first record in the sixth operation record includes the second primary key information and the data in the first record other than the first primary key information, the fifth operation record and the sixth operation record also include the identification information of the data table to which the first record belongs, and the fifth operation record and the sixth operation record correspond to the SQL statement described in the fourth operation record.

[0111] For example, the log of the source database includes the above-mentioned operation record 1, operation record 2, operation record 3 and operation record 4. Among them, operation record 4 is: update; data: ID=1, ID=2; information. The SQL statement described in operation record 4 is an update statement for updating the first primary key information (ID=1) of the record shown in Table 3 to the second primary key information (ID=2).

[0112] The database synchronization tool replaces the operation record 4 included in the log with operation records 5 and 6. The operation type in operation record 5 is delete, which means deleting the record shown in Table 3. The data belonging to the record shown in Table 3 in operation record 5 includes the first primary key information ID=1. The operation type in operation record 6 is insert, which means inserting a record. The data belonging to the record in operation record 6 includes the second primary key information ID=2 and the data in the record shown in Table 3 except the first primary key information. Operation records 5 and 6 also include the identification information of the user information table. Operation records 5 and 6 correspond to the SQL statement D described in operation record 4. Therefore, operation record 5 is represented as delete; data: ID=1; information, and operation record 6 is represented as insert; data: ID=2, name=A, age=19, career=cashier; information.

[0113] In this way, the log includes operation record 1, operation record 2, operation record 3, operation record 5 and operation record 6, and operation record 1, operation record 2, operation record 3 and operation record 5 that include the same primary key information (ID=1) and identification information (information) of the same data table are selected from the log as multiple first operation records.

[0114] Step 405: The database synchronization tool obtains a target SQL statement based on the multiple first operation records, wherein a result generated by processing the first record using the multiple SQL statements is the same as a result generated by processing the first record using the target SQL statement.

[0115] In step 405, based on the first operation type included in each first operation record and the execution order of multiple SQL statements corresponding to the multiple first operation records, the operation type included in the target SQL statement is obtained; based on the first data included in each first operation record, the data belonging to the first record included in the target SQL statement is obtained.

[0116] In some embodiments, the multiple first operation records include a first operation record 1 and a first operation record 2, the SQL statement corresponding to the first operation record 1 and the SQL statement corresponding to the first operation record 2 are two SQL statements with adjacent execution orders, the execution order of the SQL statement corresponding to the first operation record 1 is earlier than the execution order of the SQL statement corresponding to the first operation record 2, the first operation record 1 includes a first operation type 1 and first data 1 belonging to the first record, and the first operation record 2 includes a first operation type 2 and first data 2 belonging to the first record. In this way, in step 405, the following operations 4051-4053 can be used to obtain the target SQL statement.

[0117] 4051: The database synchronization tool obtains the first operation type 3 based on the first operation type 1 and the first operation type 2.

[0118] The database synchronization tool includes a mapping relationship between a first source operation type, a second source operation type, and a target operation type. The database synchronization tool uses the first operation type 1 as the first source operation type, uses the first operation type 2 as the second source operation type, and based on the first source operation type and the second source operation type, obtains the corresponding target operation type from the mapping relationship between the first source operation type, the second source operation type, and the target operation type, and uses the obtained target operation type as the first operation type 3.

[0119] For example, the multiple first operation records include the above-mentioned operation record 1, operation record 2 and operation record 3, that is, the multiple first operation records include three first operation records. The execution order of SQL statement A corresponding to the above-mentioned operation record 1 is earlier than the execution order of SQL statement B corresponding to operation record 2, and the execution order of SQL statement B corresponding to operation record 2 is earlier than the execution order of SQL statement C corresponding to operation record 3. Operation record 1 includes the first operation type insert, operation record 2 includes the first operation type update, and the database synchronization tool includes the mapping relationship between the first source operation type, the second source operation type and the target operation type as shown in Table 5 below.

[0120] Table 5

[0121] First source operation type First source operation type Target operation type insert update insert insert delete neglect update update update update delete delete delete insert update

[0122] The database synchronization tool takes the first operation type insert included in operation record 1 as the first source operation type, and the first operation type update included in operation record 2 as the second source operation type. Based on the first source operation type insert and the second source operation type update, the corresponding target operation type is obtained as insert from the mapping relationship between the first source operation type, the second source operation type and the target operation type as shown in Table 5, and the target operation type insert is taken as the first operation type 3.

[0123] In some embodiments, it is assumed that the first operation type included in the first operation record 1 is insert, indicating that the SQL statement corresponding to the first operation record 1 is used to insert the first record, and the first operation type included in the first operation record 2 is delete, indicating that the SQL statement corresponding to the first operation record 2 is used to delete the first record. The database synchronization tool uses the first operation type insert as the first source operation type and the first operation type delete as the second source operation type, and based on the first source operation type insert and the second source operation type delete, obtains the corresponding target operation type from the mapping relationship between the first source operation type, the second source operation type and the target operation type. If it is ignored, the first operation record 1 and the first operation record 2 are discarded.

[0124] Since the SQL statement corresponding to the first operation record 1 is used to save the first record, and the SQL statement corresponding to the first operation record 2 is used to delete the first record, the source database system does not change before and after executing the SQL statement corresponding to the first operation record 1 and the SQL statement corresponding to the first operation record 2 on the source database. Therefore, the target database system does not need to execute the SQL statement corresponding to the first operation record 1 and the SQL statement corresponding to the first operation record 2 on the target database, so that the first operation record 1 and the first operation record 2 can be discarded, thereby reducing the number of target SQL statements sent to the target database system to reduce the number of target SQL statements executed by the target database system.

[0125] 4052: The database synchronization tool processes the first data 1 and the first data 2 based on the first operation type 1 and the first operation type 2 to obtain the first data 3 belonging to the first record.

[0126] For example, the first data 1 belonging to the first record in the operation record 1 includes the primary key information ID=1 of the first record, the user name name=A, the user age age=18, and the user career=student. The first data 2 belonging to the first record in the operation record 2 includes the primary key information ID=1 of the first record, and the user age age=19.

[0127] In this way, based on the first operation type insert and the first operation type update, the first data 1 and the first data 2 are processed to obtain the first data 3 belonging to the first record. The first data 3 includes the primary key information ID=1 of the first record, the user name name=A, the user age age=19, and the user career career=student. The first data 3 can be expressed as (ID=1, name=A, age=19, career=student).

[0128] 4053: The database synchronization tool obtains a target SQL statement based on the first operation record 3, where the first operation record 3 includes a first operation type 3 and first data 3.

[0129] In some embodiments, the first operation record 3 also includes identification information of the data table to which the first record belongs. The first operation record 1 and the first operation record 2 both include identification information of the data table to which the first record belongs. The database synchronization tool can extract the identification information of the data table to which the first record belongs from the first operation record 1 or the first operation record 2, and generate the first operation record 3 including the first operation type 3, the identification information of the data table, and the first data 3. Then, the target SQL statement is obtained based on the first operation record 3.

[0130] In some embodiments, if the multiple SQL statements corresponding to the multiple first operation records also include the next statement whose execution order is later than the SQL statement corresponding to the first operation record 2, the first operation record 4 corresponding to the next statement is selected, that is, the SQL statement corresponding to the first operation record 4 is the next statement whose execution order is later than the SQL statement corresponding to the first operation record 2, and the first operation record 4 includes the first operation type 4 and the first data 4 belonging to the first record. In this way, in 4053, the following operations (1)-(3) can be used to obtain the target SQL statement based on the first operation record 3 and the first operation record 4. If the multiple SQL statements corresponding to the multiple first operation records do not include the next statement whose execution order is later than the SQL statement corresponding to the first operation record 2, the target SQL statement corresponding to the first operation record 3 is obtained, and the first operation record 3 is used to describe the target SQL statement. The obtained target SQL statement includes the first operation type 3, the first data 3, and the identification information of the data table to which the first record belongs.

[0131] (1): The database synchronization tool obtains the first operation type 5 based on the first operation type 3 and the first operation type 4.

[0132] The database synchronization tool uses the first operation type 3 as the first source operation type and the first operation type 4 as the second source operation type. Based on the first source operation type and the second source operation type, the tool obtains the corresponding target operation type from the mapping relationship between the first source operation type, the second source operation type and the target operation type, and uses the obtained target operation type as the first operation type 5.

[0133] For example, the database synchronization tool can extract the identification information of the data table to which the first record belongs from operation record 1 or operation record 2, and generate a first operation record 3 including the first operation type insert, the identification information of the data table and the first data 3 (ID=1, name=A, age=19, career=student).

[0134] The SQL statement C corresponding to the above operation record 3 is the next SQL statement whose execution order is later than the execution order of the SQL statement B corresponding to the operation record 2, and the operation record 3 includes the first operation type update. The database synchronization tool uses the first operation type insert included in the first operation record 3 as the first source operation type, and uses the first operation type update included in the operation record 3 as the second source operation type. Based on the first source operation type insert and the second source operation type update, the corresponding target operation type is obtained from the mapping relationship between the first source operation type, the second source operation type and the target operation type as shown in Table 5, and the target operation type insert is used as the first operation type 5.

[0135] (2): Based on the first operation type 3 and the first operation type 4, the first data 3 and the first data 4 are processed to obtain the first data 5 belonging to the first record.

[0136] For example, the first data 3 belonging to the first record includes the primary key information ID=1 of the first record, user name name=A, user age age=19, and user occupation career=student. The first data 4 belonging to the first record in the operation record 3 includes the primary key information ID=1 of the first record, and user occupation career=cashier.

[0137] In this way, the database synchronization tool processes the first data 3 and the first data 4 based on the first operation type insert and the first operation type update to obtain the first data 5 belonging to the first record. The first data 5 includes the primary key information ID=1 of the first record, the user name name=A, the user age age=19, and the user occupation career=cashier. The first data 5 can be expressed as (ID=1, name=A, age=19, career=cashier).

[0138] (3): Obtain a target SQL statement based on a first operation record 5 , where the first operation record includes a first operation type 5 and first data 5 .

[0139] In some embodiments, the first operation record 5 also includes identification information of the data table to which the first record belongs. The first operation record 3 and the first operation record 4 both include identification information of the data table to which the first record belongs. The database synchronization tool can extract the identification information of the data table to which the first record belongs from the first operation record 3 or the first operation record 4, and generate the first operation record 5 including the first operation type 5, the identification information of the data table, and the first data 5. Then, the target SQL statement is obtained based on the first operation record 5.

[0140] In some embodiments, if the multiple SQL statements corresponding to the multiple first operation records also include a next statement whose execution order is later than the SQL statement corresponding to the first operation record 4, the first operation record 6 corresponding to the next statement is selected, that is, the SQL statement corresponding to the first operation record 6 is the next statement whose execution order is later than the SQL statement corresponding to the first operation record 4. In this way, in 4053, the above operations (1)-(3) are repeatedly performed to obtain the target SQL statement based on the first operation record 5 and the first operation record 6. If the multiple SQL statements corresponding to the multiple first operation records do not include a next statement whose execution order is later than the SQL statement corresponding to the first operation record 4, the target SQL statement corresponding to the first operation record 5 is obtained, and the first operation record 5 is used to describe the target SQL statement. The obtained target SQL statement includes the first operation type 5, the first data 5, and the identification information of the data table to which the first record belongs.

[0141] For example, the first operation record 3 and the operation record 3 both include the identification information information of the data table to which the first record belongs. The database synchronization tool can extract the identification information information of the data table to which the first record belongs from the first operation record 3 and the operation record 3, and generate the first operation record 5 including the first operation type insert, the identification information information of the data table, and the first data 5 (1, A, 19, cashier). The three SQL statements corresponding to the above three first operation records do not include the next statement whose execution order is later than the SQL statement C corresponding to the operation record 3, and the target SQL statement corresponding to the first operation record 5 is obtained. The first operation record 5 is used to describe the target SQL statement. The obtained target SQL statement includes the first operation type insert, the first data 5 (1, A, 19, cashier) and the identification information information of the data table to which the first record belongs.

[0142] The target SQL statement is as follows:

[0143] insert into information(ID, name, age, career)

[0144] value(1,A,19,cashier).

[0145] Among them, the first record is processed by using SQL statement A corresponding to operation record 1, SQL statement B corresponding to operation record 2, and SQL statement C corresponding to operation record 3 in sequence, so that the first record saved in the user information table includes primary key information ID=1, user name name=A, user age age=19, and user occupation career=cashier. The first record is processed by using the target SQL statement, so that the first record saved in the user information table includes primary key information ID=1, user name name=A, user age age=19, and user occupation career=cashier. In other words, the result of processing the first record by using SQL statement A, SQL statement B, and SQL statement C in sequence is the same as the result of processing the first record by using the target SQL statement.

[0146] Step 406: The database synchronization tool sends the target SQL statement to the target database system.

[0147] Since the database synchronization tool merges multiple SQL statements for the same record into one target SQL statement, the number of SQL statements sent to the first database system is reduced, that is, the number of SQL statements that the target database system needs to execute is also reduced.

[0148] In some embodiments, the database synchronization tool may merge multiple target SQL statements, where the multiple target SQL statements are used to process different records. The database synchronization tool will send the multiple target SQL statements to the target database system.

[0149] Step 407: The target database system receives the target SQL statement and executes the target SQL statement on the target database to complete data synchronization.

[0150] In step 407, the target database driver in the target database system receives the target SQL statement and allocates a transaction to the target SQL statement. In the transaction, the target SQL statement is executed on the target database to process the target database.

[0151] In some embodiments, a target database driver in a target database system may receive multiple target SQL statements, which are used to process different records in the target database. The target database driver allocates a transaction to the multiple target SQL statements, and executes the multiple target SQL statements on the target database in the transaction to process the target database.

[0152] In order to increase the efficiency of the target database driver in processing the target database, the target database driver may process the target database based on a multi-threaded concurrent mode or characteristics of the target database.

[0153] For processing the target database based on multi-threaded concurrent mode, the target database driver can process the target database in the following ways.

[0154] Method 1 is concurrency by table, that is, for at least one target SQL statement for processing records in the same data table, the at least one target SQL statement is placed in the same thread, and the thread executes the at least one target SQL statement to process the data table in the target database. In this way, multiple data tables can be processed simultaneously through multiple threads, thereby improving the efficiency of processing the target database.

[0155] Each target SQL statement includes identification information of a data table. In the first mode, the target database driver can select at least one target SQL statement including identification information of the same data table from the received multiple target SQL statements, and assign the at least one target SQL statement to the same thread, so that the thread executes the at least one target SQL statement. That is, the thread processes the data table in the target database based on the at least one target SQL statement.

[0156] Mode 2 is concurrent execution based on primary key information, that is, for at least one target SQL statement for processing the same record in the same data table, the data table is a data table in the target database, the at least one target SQL statement is put into the same thread, and the thread executes the at least one target SQL statement to process the record in the data table. In this way, multiple records in the target database can be processed simultaneously through multiple threads, thereby improving the efficiency of processing the target database.

[0157] Each target SQL statement includes the identification information of the data table and the primary key information of a record in the data table. In the second mode, the target database driver can select at least one target SQL statement including the identification information of the same data table and the same primary key information from the received multiple target SQL statements, and assign the at least one target SQL statement to the same thread, so that the thread executes the at least one target SQL statement. That is, for the record in the data table including the primary key information, the thread processes the record based on the at least one target SQL statement.

[0158] To process the target database based on the characteristics of the target database, the target database driver can process the target database in the following ways.

[0159] Mode 1 is a transaction merging mode. Each time the target database driver receives a batch of target SQL statements, it allocates a transaction to the target SQL statements. The batch of target SQL statements includes one or more SQL statements. There may be multiple transactions to be executed in the target database driver. Each transaction corresponds to at least one target SQL statement. Transactions whose number of corresponding target SQL statements is less than the quantity threshold are defined as small transactions. The target database driver can merge multiple small transactions into a large transaction, and submit the target SQL statements corresponding to the large transaction to the target database. The target SQL statements corresponding to the large transaction include the target SQL statements corresponding to each small transaction. The target database executes the target SQL statements corresponding to the large transaction in the large transaction, thereby reducing the number of times the target database driver submits the target SQL statements to the target database, thus saving the time required to submit SQL statements and improving the efficiency of processing the target database.

[0160] Mode 2 is batch mode, and the target database provides a batch write interface. The target database driver can submit at least one target SQL statement including the same operation type to the target database through the batch write interface. The target database executes the at least one target SQL statement, thereby reducing the number of times the target database driver submits the target SQL statement to the target database, thus saving the time required to submit SQL statements and improving the efficiency of processing the target database.

[0161] Mode 3 is the SQL statement concatenation mode. The target database driver can concatenate multiple target SQL statements into a string and submit the string to the target database. The target database receives the string, parses the string to obtain the multiple target SQL statements, and executes the multiple target SQL statements, thereby reducing the number of times the target database driver submits the target SQL statements to the target database, thus saving the time required to provide SQL statements and improving the efficiency of processing the target database.

[0162] In an embodiment of the present application, a database synchronization tool selects multiple first operation records including identification information of the same data table and the same primary key information from the log of the source database, and the multiple SQL statements corresponding to the multiple first operation records are multiple statements for processing the same record. The database synchronization tool merges the multiple SQL statements into a target SQL statement based on the multiple first operation records, and the result generated by processing the record using the multiple SQL statements is the same as the result generated by processing the record using the target SQL statement. The database synchronization tool sends the target SQL statement to the target database system, and the target database system executes the target SQL statement on the target database to achieve data synchronization. Among them, if the database synchronization tool sends the multiple SQL statements to the target database system, the cumulative value of the time consumed by the target database system to execute the multiple SQL statements on the target database is much greater than the time consumed by the target database system to execute the target SQL statement on the target database, so the database synchronization tool merges the multiple SQL statements into a target SQL statement, reducing the number of SQL statements executed by the target database system on the target database, thereby improving the efficiency of data synchronization. Since the number of SQL statements sent by the database synchronization tool to the target database system is reduced, the occupation of network resources is also reduced.

[0163] See also Figure 5 The present application embodiment provides a data synchronization device 500, which is used to synchronize data in a source database to a target database. The device 500 can be deployed in the database synchronization tool in any of the above embodiments. For example, the device 500 can be deployed in Figure 3 In the database synchronization tool 200 in the network architecture 20 shown in the figure, or, the device 500 can be deployed in Figure 4 In the database synchronization tool of the method 400 shown. The device 500 includes:

[0164] Processing unit 501 is used to obtain multiple first operation records, where the multiple first operation records correspond to multiple structured query language SQL statements executed by the source database system on the source database, each first operation record is used to describe the SQL statement corresponding to each first operation record, and the SQL statement corresponding to each first operation record is a statement used to process the first record;

[0165] The processing unit 501 is further configured to obtain a target SQL statement based on the multiple first operation records, wherein a result generated by processing the first record using the multiple SQL statements is the same as a result generated by processing the first record using the target SQL statement;

[0166] The sending unit 502 is used to send a target SQL statement to a target database system, and the target database system is used to execute the target SQL statement on the target database to complete data synchronization.

[0167] Optionally, the detailed implementation process of the processing unit 501 obtaining multiple first operation records can be found in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0168] Optionally, the detailed implementation process of the processing unit 502 obtaining the target SQL statement based on the multiple first operation records is shown in Figure 4 The relevant contents of step 405 of the method 400 are not described in detail here.

[0169] Optionally, the detailed implementation process of the sending unit 502 sending the target SQL statement to the target database system is shown in Figure 4 The relevant contents of step 406 of the method 400 are not described in detail here.

[0170] Optionally, the first operation record includes a first operation type and first data belonging to the first record, and the first operation type is an operation type of the first record processed by a SQL statement corresponding to the first operation record.

[0171] Optionally, the processing unit 501 is configured to:

[0172] Based on the first operation type included in each first operation record and the execution order of multiple SQL statements corresponding to the multiple first operation records, obtain the operation type included in the target SQL statement;

[0173] Based on the first data included in each first operation record, the data belonging to the first record included in the target SQL statement is obtained.

[0174] Optionally, the processing unit 501 obtains the detailed implementation process of the operation type included in the target SQL statement. Figure 4 The relevant contents of step 4051 of the method 400 are not described in detail here.

[0175] Optionally, the processing unit 502 obtains the data belonging to the first record included in the target SQL statement. Figure 4 The relevant contents of step 4052 of the method 400 are not described in detail here.

[0176] Optionally, the processing unit 501 is further configured to:

[0177] Acquire a second operation record and a third operation record, wherein the second operation record includes a second operation type and second data belonging to the second record, and the third operation record includes a third operation type and third data belonging to the second record;

[0178] When the second operation type is inserting the second record and the third operation type is deleting the second record, the second operation record and the third operation record are discarded.

[0179] Optionally, the detailed implementation process of the processing unit 501 obtaining the second operation record and the third operation record is shown in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0180] Optionally, the detailed implementation process of the processing unit 502 discarding the second operation record and the third operation record is shown in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0181] Optionally, the first operation record further includes identification information of a data table to which the first record belongs, the first data belonging to the first record includes primary key information of the first record, and the data table is a data table in a source database;

[0182] The processing unit 501 is configured to:

[0183] Obtain a log of a source database, where the log includes multiple operation records;

[0184] A plurality of operation records including the same primary key information and identification information of the same data table are selected from the log as a plurality of first operation records.

[0185] Optionally, the detailed implementation process of the processing unit 501 obtaining the log of the source database can be found in Figure 4 The relevant contents of step 403 of the method 400 are not described in detail here.

[0186] Optionally, the processing unit 502 selects multiple operation records including the same primary key information and the identification information of the same data table from the log as multiple first operation records. For a detailed implementation process, see Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0187] Optionally, the multiple operation records include a fourth operation record, and the SQL statement described in the fourth operation record is used to update the first primary key information of the first record to the second primary key information. The processing unit 501 is further used to:

[0188] The fourth operation record included in the log is replaced with the fifth operation record and the sixth operation record, the operation type in the fifth operation record is to delete the first record, and the data belonging to the first record in the fifth operation record includes the first primary key information; the operation type in the sixth operation record is to insert the first record, and the data belonging to the first record in the sixth operation record includes the second primary key information and the data in the first record except the first primary key information; the fifth operation record and the sixth operation record correspond to the SQL statement described in the fourth operation record.

[0189] Optionally, the processing unit 502 replaces the fourth operation record included in the log with the fifth operation record and the sixth operation record. Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0190] In the embodiment of the present application, the processing unit obtains multiple first operation records, the multiple SQL statements corresponding to the multiple first operation records are multiple SQL statements for processing the first records, and the target SQL statement is obtained based on the multiple first operation records, wherein the result generated by processing the first record using the multiple SQL statements is the same as the result generated by processing the first record using the target SQL statement. This reduces the number of SQL statements sent by the sending unit to the target database system, and reduces the number of SQL statements executed by the target database system on the target database, thereby improving the efficiency of data synchronization.

[0191] See also Figure 6 , the present application embodiment provides a computing device 600. For example, the computing device 600 may be Figure 3 The network architecture 20 shown includes a database synchronization tool. For example, the computing device 600 may be the database synchronization tool 200 in the network architecture 20, or the computing device 600 may be Figure 4 The database synchronization tool in the method 400 is shown, etc.

[0192] like Figure 6 As shown, computing device 600 includes: bus 602, processor 604, memory 606 and communication interface 608. Processor 604, memory 606 and communication interface 608 communicate through bus 602. Computing device 600 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 600.

[0193] The bus 602 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 602 may include a path for transmitting information between various components of the computing device 600 (eg, the processor 604, the memory 606, and the communication interface 608).

[0194] The processor 604 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0195] The memory 606 may include a volatile memory, such as a random access memory (RAM). The memory 606 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0196] See also Figure 6 , the memory 606 stores executable program codes, and the processor 604 executes the executable program codes to respectively implement Figure 5 The functions of the processing unit 501 and the sending unit 502 in the device 500 shown in the figure are implemented to realize the method provided by any of the above embodiments. That is, the memory 606 stores instructions for executing the method provided by any of the above embodiments. Or,

[0197] The communication interface 608 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or communication networks.

[0198] The embodiment of the present application also provides a data synchronization cluster. The data synchronization cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0199] like Figure 7 As shown, the data synchronization cluster includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the data synchronization cluster may store the same instructions for executing the method provided in any of the above embodiments.

[0200] In some possible implementations, the memory 606 of one or more computing devices 600 in the data synchronization cluster may also store partial instructions for executing the above data synchronization method. In other words, a combination of one or more computing devices 600 may jointly execute instructions for executing the method provided in any of the above embodiments.

[0201] In some possible implementations, one or more computing devices in the data synchronization cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0202] In this type of possible implementation, the memory 606 in the computing device 600A stores the execution Figure 5 Instructions for the functions of the processing unit 501 in the illustrated embodiment. Meanwhile, the memory 606 in the computing device 600B stores instructions for executing the functions of the processing unit 501 in the illustrated embodiment. Figure 5 Instructions for the functionality of the sending unit 502 in the illustrated embodiment.

[0203] It should be understood that Figure 8 The functions of the computing device 600A shown in FIG. 6A may also be completed by multiple computing devices 600. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices 600.

[0204] The present application embodiment also provides another data synchronization cluster. The connection relationship between the computing devices in the data synchronization cluster can be similarly referred to as Figure 8 The connection mode of the data synchronization cluster is different in that the memory 606 in one or more computing devices 600 in the data synchronization cluster may store the same instructions for executing the method provided in any of the above embodiments.

[0205] In some possible implementations, the memory 606 of one or more computing devices 600 in the data synchronization cluster may also store partial instructions for executing the method provided in any of the above embodiments. In other words, the combination of one or more computing devices 600 may jointly execute instructions for executing the method provided in any of the above embodiments.

[0206] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method provided in any of the above embodiments.

[0207] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the method provided in any of the above embodiments.

[0208] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0209] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for data synchronization, characterized in that: The method is performed by a database synchronization tool, and the database synchronization tool is used to synchronize data in a source database to a target database. The method includes: Acquire a plurality of first operation records, the plurality of first operation records corresponding to a plurality of structured query language SQL statements executed by the source database system on the source database, each first operation record being used to describe the SQL statement corresponding to each first operation record, and the SQL statement corresponding to each first operation record being a statement for processing the first record; Acquire a target SQL statement based on the multiple first operation records, wherein a result generated by processing the first records using the multiple SQL statements is the same as a result generated by processing the first record using the target SQL statement; The target SQL statement is sent to a target database system, and the target database system is used to execute the target SQL statement on the target database to complete data synchronization.

2. The method according to claim 1, characterized in that The first operation record includes a first operation type and first data belonging to the first record, and the first operation type is an operation type of the SQL statement corresponding to the first operation record for processing the first record.

3. The method according to claim 2, characterized in that The acquiring a target SQL statement based on the multiple first operation records includes: Based on the first operation type included in each first operation record and the execution order of multiple SQL statements corresponding to the multiple first operation records, obtaining the operation type included in the target SQL statement; Based on the first data included in each first operation record, the data included in the target SQL statement and belonging to the first record is obtained.

4. The method according to claim 2 or 3, characterized in that The method further comprises: Acquire a second operation record and a third operation record, wherein the second operation record includes a second operation type and second data belonging to the second record, and the third operation record includes a third operation type and third data belonging to the second record; When the second operation type is inserting the second record and the third operation type is deleting the second record, the second operation record and the third operation record are discarded.

5. The method according to any one of claims 2 to 4, characterized in that: The first operation record further includes identification information of a data table to which the first record belongs, the first data belonging to the first record includes primary key information of the first record, and the data table is a data table in the source database; The obtaining of a plurality of first operation records comprises: Obtaining a log of the source database, the log including a plurality of operation records; A plurality of operation records including the same primary key information and identification information of the same data table are selected from the log as the plurality of first operation records.

6. The method according to claim 5, characterized in that The multiple operation records include a fourth operation record, and the SQL statement described in the fourth operation record is used to update the first primary key information of the first record to the second primary key information. Before selecting multiple operation records including the same primary key information and the identification information of the same data table from the log, the method further includes: The fourth operation record included in the log is replaced with a fifth operation record and a sixth operation record, the operation type in the fifth operation record is to delete the first record, the data belonging to the first record in the fifth operation record includes the first primary key information, the operation type in the sixth operation record is to insert the first record, the data belonging to the first record in the sixth operation record includes the second primary key information and the data in the first record except the first primary key information, and the fifth operation record and the sixth operation record correspond to the SQL statement described in the fourth operation record.

7. A data synchronization device, characterized in that: The device is used to synchronize data in a source database to a target database, and the device includes: a processing unit, configured to obtain a plurality of first operation records, wherein the plurality of first operation records correspond to a plurality of structured query language SQL statements executed by a source database system on the source database, each first operation record is respectively used to describe the SQL statement corresponding to each first operation record, and the SQL statement corresponding to each first operation record is a statement used to process the first record; The processing unit is further configured to obtain a target SQL statement based on the multiple first operation records, wherein a result generated by processing the first record using the multiple SQL statements is the same as a result generated by processing the first record using the target SQL statement; The sending unit is used to send the target SQL statement to the target database system, and the target database system is used to execute the target SQL statement on the target database to complete data synchronization.

8. The device according to claim 7, characterized in that The first operation record includes a first operation type and first data belonging to the first record, and the first operation type is an operation type of the SQL statement corresponding to the first operation record for processing the first record.

9. The device according to claim 8, characterized in that The processing unit is used for: Based on the first operation type included in each first operation record and the execution order of multiple SQL statements corresponding to the multiple first operation records, obtaining the operation type included in the target SQL statement; Based on the first data included in each first operation record, the data included in the target SQL statement and belonging to the first record is obtained.

10. The device according to claim 8 or 9, characterized in that The processing unit is further used for: Acquire a second operation record and a third operation record, wherein the second operation record includes a second operation type and second data belonging to the second record, and the third operation record includes a third operation type and third data belonging to the second record; When the second operation type is inserting the second record and the third operation type is deleting the second record, the second operation record and the third operation record are discarded.

11. The device according to any one of claims 8 to 10, characterized in that: The first operation record further includes identification information of a data table to which the first record belongs, the first data belonging to the first record includes primary key information of the first record, and the data table is a data table in the source database; The processing unit is used for: Obtaining a log of the source database, the log including a plurality of operation records; A plurality of operation records including the same primary key information and identification information of the same data table are selected from the log as the plurality of first operation records.

12. The device according to claim 11, characterized in that The multiple operation records include a fourth operation record, wherein the SQL statement described in the fourth operation record is used to update the first primary key information of the first record to the second primary key information, and the processing unit is further used to: The fourth operation record included in the log is replaced with a fifth operation record and a sixth operation record, the operation type in the fifth operation record is to delete the first record, the data belonging to the first record in the fifth operation record includes the first primary key information, the operation type in the sixth operation record is to insert the first record, the data belonging to the first record in the sixth operation record includes the second primary key information and the data in the first record except the first primary key information, and the fifth operation record and the sixth operation record correspond to the SQL statement described in the fourth operation record.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

15. A computer program product comprising instructions, characterized in that When the instruction is executed by the computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

Images