Data synchronization method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose a data synchronization method, device, electronic device, and computer-readable medium. A specific implementation of the method includes: in response to a successful connection with a target database, receiving data synchronization task type information; obtaining the to-be-synchronized data corresponding to the data synchronization task type information from the target database; synchronizing the to-be-synchronized data to a backup database according to a concurrent execution type, wherein the backup database is used to back up the data in the target database, and the concurrent execution type characterizes the way to synchronize the to-be-synchronized data to the backup database. This implementation improves data synchronization efficiency and reduces the waste of bandwidth resources.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a data synchronization method, device, electronic device, and computer-readable medium. Background Art

[0002] With the popularization and development of computer technology, the data generated by network applications has increased explosively. The existing technology often uses distributed databases to store massive data. In order to avoid data loss or damage caused by a failure of the distributed database, it is often necessary to synchronize the data in the database to be backed up to the backup database. At present, the commonly used method is: obtain the data to be backed up from the database to be backed up, and synchronize the data to be backed up to the backup database.

[0003] However, when using the above method for data synchronization, the following technical problems often occur:

[0004] Often, a data synchronization method that matches the data synchronization scenario is not selected according to the data synchronization scenario, resulting in low data synchronization efficiency and, in turn, a waste of bandwidth resources. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a data synchronization method, an apparatus, an electronic device, and a computer-readable medium to solve one of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a data synchronization method, the method comprising: in response to a successful connection with a target database, receiving data synchronization task type information; obtaining the data to be synchronized corresponding to the data synchronization task type information from the target database; and synchronizing the data to be synchronized to a backup database according to a concurrent execution type, wherein the backup database is used to back up the data in the target database, and the concurrent execution type represents a manner of synchronizing the data to be synchronized to the backup database.

[0008] Optionally, the obtaining of the data to be synchronized corresponding to the data synchronization task type information from the target database includes: in response to determining that the data synchronization task type information is the first data synchronization type, obtaining full data from the target database as the data to be synchronized.

[0009] Optionally, the obtaining of the data to be synchronized corresponding to the data synchronization task type information from the target database includes: in response to determining that the data synchronization task type information is the second data synchronization type, obtaining incremental data from the target database as the data to be synchronized.

[0010] Optionally, the obtaining of the data to be synchronized corresponding to the data synchronization task type information from the target database includes: in response to determining that the data synchronization task type information is a third data synchronization type, obtaining full data and incremental data from the target database as the data to be synchronized.

[0011] Optionally, synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: in response to determining that the concurrent execution type is the first concurrent execution type, synchronizing each record in the data to be synchronized to the backup database in a serial manner.

[0012] Optionally, synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: in response to determining that the concurrent execution type is the second concurrent execution type, generating at least one database operation statement based on each record in the data to be synchronized; and executing each of the at least one database operation statement.

[0013] Optionally, the at least one database statement includes: at least one database insert statement; and generating at least one database operation statement based on each record in the data to be synchronized, including: in response to determining that the data to be synchronized includes full data, converting each record in the full data into a database insert statement to obtain at least one database insert statement.

[0014] Optionally, the at least one database statement further includes: at least one sub-database operation statement; and the generation of at least one database operation statement based on each record in the data to be synchronized further includes: in response to determining that the data to be synchronized includes incremental data, generating candidate sub-database operation statements according to the event type corresponding to each record in the incremental data to obtain at least one candidate sub-database operation statement; in response to determining that the target data table satisfies the first parallel processing condition, merging the candidate sub-database operation statements that meet the merge condition in the at least one candidate sub-database operation statement to generate a sub-database operation statement to obtain the at least one sub-database operation statement, wherein the target data table is the data table corresponding to the data to be synchronized, the first parallel processing condition is that the target data table includes a unique index or a primary key, and the merge condition is that the primary key or unique index of the data table operated by the candidate sub-database operation statements is the same.

[0015] Optionally, the execution of each of the at least one database operation statement includes: in response to determining that the target data table satisfies a second parallel processing condition, executing each of the at least one database operation statement in a serial working mode, wherein the second parallel processing condition is that the target data table contains a unique index and a primary key.

[0016] Optionally, synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: in response to determining that the concurrent execution type is the third concurrent execution type, performing transaction parsing on the data to be synchronized to generate at least one pending transaction; determining the record information group corresponding to each pending transaction in the at least one pending transaction to obtain a record information group set; selecting a predetermined number of pending transactions from the at least one pending transaction as target pending transactions to obtain a target pending transaction set; performing transaction conflict detection on the target pending transactions according to the record information group corresponding to each target pending transaction in the target pending transaction set to generate transaction conflict detection information to obtain a transaction conflict detection information set; generating a directed acyclic graph based on the target pending transaction set and the transaction conflict detection information set, wherein the nodes in the directed acyclic graph correspond one-to-one to the target pending transactions in the target pending transaction set; topologically sorting the directed acyclic graph to generate a topologically sorted sequence; and executing the target pending transactions corresponding to each node in the topologically sorted sequence.

[0017] In a second aspect, some embodiments of the present disclosure provide a data synchronization device, comprising: a receiving unit, configured to receive data synchronization task type information in response to a successful connection with a target database; an acquisition unit, configured to acquire the data to be synchronized corresponding to the data synchronization task type information from the above-mentioned target database; a synchronization unit, configured to synchronize the above-mentioned data to be synchronized to a backup database according to a concurrent execution type, wherein the above-mentioned backup database is used to back up the data in the above-mentioned target database, and the above-mentioned concurrent execution type represents a manner of synchronizing the above-mentioned data to be synchronized to the above-mentioned backup database.

[0018] Optionally, the acquisition unit is further configured to: in response to determining that the data synchronization task type information is the first data synchronization type, acquire full data from the target database as the data to be synchronized.

[0019] Optionally, the acquisition unit is further configured to: in response to determining that the data synchronization task type information is the second data synchronization type, acquire incremental data from the target database as the data to be synchronized.

[0020] Optionally, the acquisition unit is further configured to: in response to determining that the data synchronization task type information is a third data synchronization type, acquire full data and incremental data from the target database as the data to be synchronized.

[0021] Optionally, the synchronization unit is further configured to: in response to determining that the concurrent execution type is the first concurrent execution type, synchronize each record in the to-be-synchronized data to the backup database in a serial manner.

[0022] Optionally, the synchronization unit is further configured to: in response to determining that the concurrent execution type is the second concurrent execution type, generate at least one database operation statement based on each record in the data to be synchronized; and execute each of the at least one database operation statement.

[0023] Optionally, the at least one database statement includes: at least one database insert statement; and the synchronization unit is further configured to: in response to determining that the data to be synchronized includes full data, convert each record in the full data into a database insert statement to obtain at least one database insert statement.

[0024] Optionally, the at least one database statement further includes: at least one sub-database operation statement; and the synchronization unit is further configured to: in response to determining that the data to be synchronized includes incremental data, generate a candidate sub-database operation statement according to the event type corresponding to each record in the incremental data, and obtain at least one candidate sub-database operation statement; in response to determining that the target data table satisfies the first parallel processing condition, merge the candidate sub-database operation statements that meet the merge condition in the at least one candidate sub-database operation statement to generate a sub-database operation statement, and obtain the at least one sub-database operation statement, wherein the target data table is the data table corresponding to the data to be synchronized, the first parallel processing condition is that the target data table includes a unique index or a primary key, and the merge condition is that the primary key or unique index of the data table operated by the candidate sub-database operation statement is the same.

[0025] Optionally, the synchronization unit is further configured to: in response to determining that the target data table satisfies a second parallel processing condition, execute each of the at least one database operation statement in a serial working manner, wherein the second parallel processing condition is that the target data table contains a unique index and a primary key.

[0026] Optionally, the synchronization unit is further configured to: in response to determining that the above-mentioned concurrent execution type is the third concurrent execution type, perform transaction parsing on the data to be synchronized to generate at least one pending transaction; determine the record information group corresponding to each pending transaction in the above-mentioned at least one pending transaction to obtain a record information group set; select a predetermined number of pending transactions from the above-mentioned at least one pending transaction as target pending transactions to obtain a target pending transaction set; perform transaction conflict detection on the above-mentioned target pending transactions according to the record information group corresponding to each target pending transaction in the above-mentioned target pending transaction set to generate transaction conflict detection information to obtain a transaction conflict detection information set; generate a directed acyclic graph based on the above-mentioned target pending transaction set and the above-mentioned transaction conflict detection information set, wherein the nodes in the above-mentioned directed acyclic graph correspond one-to-one to the target pending transactions in the above-mentioned target pending transaction set; perform topological sorting on the above-mentioned directed acyclic graph to generate a topological sorting sequence; and execute the target pending transactions corresponding to each node in the above-mentioned topological sorting sequence.

[0027] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0028] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0029] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the data synchronization method of some embodiments of the present disclosure, data synchronization tasks in various data synchronization scenarios are met, and the data synchronization efficiency is improved. Specifically, the reason for the low data synchronization efficiency is that: the data synchronization method matching the data synchronization scenario is often not selected according to the data synchronization scenario. Based on this, the data synchronization method of some embodiments of the present disclosure, first, in response to the successful connection with the target database, receives the data synchronization task type information. In actual situations, when the data in the above-mentioned target database is synchronized and backed up, it is often necessary to ensure that the communication connection between the above-mentioned target database and the backup database is successful. When the connection is successful, the data synchronization task type information is obtained again, thereby avoiding the situation of data synchronization failure caused by connection failure. Secondly, the data to be synchronized corresponding to the above-mentioned data synchronization task type information is obtained from the above-mentioned target database. According to the data synchronization task type information, the corresponding data to be synchronized is obtained, thereby reducing the data synchronization pressure caused by directly synchronizing all the data in the target database. Finally, according to the concurrent execution type, the above-mentioned data to be synchronized is synchronized to the backup database. In actual situations, the amount of data that needs to be synchronized for different data synchronization tasks is often different. By flexibly selecting the data synchronization method through concurrent execution types, data synchronization efficiency is improved and the waste of bandwidth resources is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0031] Figure 1 is a schematic diagram of an application scenario of the data synchronization method of some embodiments of the present disclosure;

[0032] Figure 2 is a flow chart of some embodiments of the data synchronization method according to the present disclosure;

[0033] Figure 3 are flow charts of other embodiments of the data synchronization method according to the present disclosure;

[0034] Figure 4 is a schematic diagram of a target graph in the data synchronization method of the present disclosure;

[0035] Figure 5 is a schematic diagram of a directed acyclic graph in the data synchronization method of the present disclosure;

[0036] Figure 6 is a schematic diagram of a topological sorting sequence in the data synchronization method disclosed in the present invention;

[0037] Figure 7 is a schematic diagram of the structure of some embodiments of the data synchronization device according to the present disclosure;

[0038] Figure 8 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0039] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0040] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0041] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0042] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0043] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0044] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0045] Figure 1 It is a schematic diagram of an application scenario of the data synchronization method of some embodiments of the present disclosure.

[0046] exist Figure 1In the application scenario, first, the computing device 101 can receive data synchronization task type information 103 (for example, the data synchronization task type information 103 can indicate a multi-table synchronization type) in response to a successful connection with the target database 102. Then, the computing device 101 can obtain the to-be-synchronized data 104 corresponding to the data synchronization task type information 103 from the target database 102. Finally, the computing device 101 can synchronize the to-be-synchronized data 104 to the backup database 106 according to the concurrent execution type 105 (for example, the "asynchronous replication" type). The backup database 106 is used to back up the data in the target database 102. The concurrent execution type 105 can be used to characterize the manner in which the to-be-synchronized data 104 is synchronized to the backup database 106.

[0047] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.

[0048] It should be understood that Figure 1 The number of computing devices in the embodiment is only illustrative. Any number of computing devices may be provided according to implementation requirements.

[0049] Continue to refer Figure 2 , shows a process 200 of some embodiments of the data synchronization method according to the present disclosure. The data synchronization method comprises the following steps:

[0050] Step 201: In response to a successful connection with a target database, data synchronization task type information is received.

[0051] In some embodiments, the execution body of the data synchronization method (eg Figure 1The computing device 101 shown in the figure can receive data synchronization task type information in response to successful connection with the target database by wired connection or wireless connection. The target database can be a database that needs to be synchronized with data. The target database can be a distributed database. The target database can also be a non-distributed database. For example, the target database can be a MongoDB database. The target database can also be a MySQL database. The data synchronization task type information can be used to characterize the type of data synchronization task. The data synchronization task type information can be manually configured. The data synchronization task type information can be a "single table synchronization type". The data synchronization task type information can also be a "multi-table synchronization type". The "single table synchronization type" can represent data synchronization of data in the target database in units of one data table. The "multi-table synchronization type" can represent data synchronization of data in the target database in units of a target number of data tables. The execution subject can respond to receiving a database connection success signal to determine that the connection with the target database is successful. The database connection success signal can be "200". The "200" can represent a successful connection with the target database. The data synchronization scenario may represent the task type of data synchronization.

[0052] As an example, the above data synchronization scenario may be a "single-table synchronization type". The above data synchronization scenario may also be a "multi-table synchronization type".

[0053] Optionally, the above-mentioned data synchronization task type information may also be not limited to one of the following: "first data synchronization type", "second data synchronization type" and "third data synchronization type". Among them, the above-mentioned "first data synchronization type" may represent the data synchronization of the full data in the above-mentioned target database. The above-mentioned "second data synchronization type" may represent the data synchronization of the incremental data in the above-mentioned target database. The above-mentioned "third data synchronization type" may represent the data synchronization of the full data and incremental data in the above-mentioned target database. The above-mentioned full data may refer to all the data stored in the above-mentioned target database. The above-mentioned incremental data may refer to the new data stored in the above-mentioned target database after the previous data synchronization.

[0054] As another example, the data synchronization scenario may be a “first data synchronization type”. The data synchronization scenario may also be a “second data synchronization type”. The data synchronization scenario may also be a “third data synchronization type”.

[0055] Step 202: Acquire the data to be synchronized corresponding to the data synchronization task type information from the target database.

[0056] In some embodiments, the execution subject obtains the to-be-synchronized data corresponding to the data synchronization task type information from the target database, which may include the following steps:

[0057] The first step is to obtain the table information of all data tables stored in the above target database to obtain a table information sequence.

[0058] The table information in the above table information sequence may include: a data table name.

[0059] As an example, the execution subject may obtain the table information of all data tables stored in the target database through the following SQL (Structured Query Language) statement:

[0060] SELECT TALBE_NAME

[0061] FROM information_schema."TABLES"

[0062] WHERE TABLE_SCHEMA="db_name".

[0063] The “db_name” mentioned above may be the name of the target database mentioned above.

[0064] As another example, the above table information sequence may be ["table1", "table2", "table3", "table4"].

[0065] In the second step, according to the data synchronization task type information and the table information sequence, the to-be-synchronized data corresponding to the data synchronization task type information is obtained from the target database through SQL statements.

[0066] The data to be synchronized may represent data that needs to be synchronized in the target database. The data to be synchronized may be at least one row of data in the target database. The data to be synchronized may also be at least one transaction information in a binary log file corresponding to the target database. The binary log file may be a MySQLBinlog log file. The transaction corresponding to the transaction information may be a database transaction.

[0067] When the data synchronization task type information is "single table synchronization type". First, the execution subject can generate a first query statement according to each table information in the table information sequence to obtain a first query statement sequence. Then, the data to be synchronized can be obtained according to each first query statement in the first query statement sequence.

[0068] When the data synchronization task type information is "multi-table synchronization type". First, the execution subject can generate a second query statement based on the table information in the table information sequence. Then, the data to be synchronized can be obtained according to the second query statement. Among them, the first query statement in the first query statement sequence can be an "INSERT" type statement in an SQL statement. The second query statement can be an "INSERT" type statement in an SQL statement. As an example, the table information sequence can be ["table1", "table2", "table3", "table4"]. The data synchronization task type information can be "single table synchronization type". The first query statement can be "SELECT*FROM table1". The execution subject can obtain the data to be synchronized through "SELECT*FROM table1". The data to be synchronized can be "[record 1, record 2]"

[0069] As another example, the data synchronization task type information may be "multi-table synchronization type". The target number may be "2". The table information sequence may be ["table1", "table2", "table3", "table4"]. The second query statement may be "SELECT * FROM table1, table2". The execution subject may obtain the data to be synchronized through "SELECT * FROM table1, table2". The data to be synchronized may be "[record 1, record 2, record 3, record 4]".

[0070] In some optional implementations of some embodiments, the execution subject may, in response to determining that the data synchronization task type information is the first data synchronization type, obtain the full amount of data from the target database as the data to be synchronized through an SQL statement.

[0071] As an example, the execution subject may obtain the full amount of data as the data to be synchronized through the SQL statement "SELECT * FROM data table name".

[0072] In some optional implementations of some embodiments, the execution subject may, in response to determining that the data synchronization task type information is the second data synchronization type, obtain incremental data from the target database as data to be synchronized.

[0073] The execution subject may obtain incremental data from the target database as data to be synchronized through SQL statements according to the first target time and the second target time. The first target time may be the time when the previous data synchronization was completed. The second target time may be the current time.

[0074] As an example, the execution subject may obtain incremental data from the target database as the data to be synchronized through the following SQL statement:

[0075] SELECT *

[0076] FROM table name

[0077] WHERE timestamp BETWEEN first target time AND second target time.

[0078] In some optional implementations of some embodiments, in response to determining that the data synchronization task type information is the third data synchronization type, the execution subject may obtain full data and incremental data from the target database as the data to be synchronized through the following steps:

[0079] The first step is to obtain the above full data from the above target database through SQL statements.

[0080] As an example, the above execution subject can obtain the above full data through the SQL statement "SELECT * FROM data table name".

[0081] The second step is to obtain all records stored in the temporary data table as incremental data.

[0082] Among them, the above-mentioned execution entity can create a trigger. When a new record is generated, the trigger can store the new record in the above-mentioned target database and also store the new record in the above-mentioned temporary data table. When the above-mentioned execution entity obtains the incremental data of the above-mentioned target database, the above-mentioned execution entity can directly determine the records stored in the above-mentioned temporary data table as the above-mentioned incremental data. When the amount of data in the above-mentioned target database is large, this method can reduce the frequency of reading and writing to the above-mentioned target database and reduce the reading and writing pressure of the above-mentioned target database.

[0083] Step 203: synchronize the data to be synchronized to the backup database according to the concurrent execution type.

[0084] In some embodiments, the execution subject may synchronize the data to be synchronized to the backup database by wired connection or wireless connection according to the concurrent execution type. The concurrent execution type may characterize the way to synchronize the data to be synchronized to the backup database. The backup database may be a database for backing up the data in the target database. The backup database may be a distributed database. The backup database may also be a non-distributed database. For example, the backup database may be a MongoDB database. The backup database may also be a MySQL database. The concurrent execution type may be generated by manual configuration. The execution subject may also determine the concurrent execution type according to the amount of data to be synchronized. The concurrent execution type may be an "asynchronous replication" type. The concurrent execution type may also be a "synchronous replication" type. When the amount of data to be synchronized is less than a predetermined size, the execution subject may adopt the "synchronous replication" type to synchronize the data to be synchronized to the backup database. When the amount of data to be synchronized is greater than or equal to the predetermined size, the execution subject may adopt the "asynchronous replication" type to synchronize the data to be synchronized to the backup database. The predetermined size may be 200 megabytes. Here, there is no limitation on the setting of the predetermined size.

[0085] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the data synchronization method of some embodiments of the present disclosure, data synchronization tasks in various data synchronization scenarios are met, and the data synchronization efficiency is improved. Specifically, the reason for the low data synchronization efficiency is that: the data synchronization method that matches the data synchronization scenario is often not selected according to the data synchronization scenario. Based on this, the data synchronization method of some embodiments of the present disclosure, first, in response to the successful connection with the target database, receives the data synchronization task type information. In actual situations, when the data in the above-mentioned target database is synchronized and backed up, it is often necessary to ensure that the communication connection between the above-mentioned target database and the backup database is successful. When the connection is successful, the data synchronization task type information is obtained again, thereby avoiding the situation of data synchronization failure caused by connection failure. Secondly, the data to be synchronized corresponding to the above-mentioned data synchronization task type information is obtained from the above-mentioned target database. According to the data synchronization task type information, the corresponding data to be synchronized is obtained, thereby reducing the data synchronization pressure caused by directly synchronizing all the data in the target database. Finally, according to the concurrent execution type, the above-mentioned data to be synchronized is synchronized to the backup database. In actual situations, different data synchronization tasks often have different amounts of data to be synchronized. Through concurrent execution types, data synchronization methods can be flexibly selected to improve data synchronization efficiency and reduce bandwidth resource consumption.

[0086] Further references Figure 3, which shows a process 300 of another embodiment of a data synchronization method. The process 300 of the data synchronization method includes the following steps:

[0087] Step 301: In response to a successful connection with a target database, data synchronization task type information is received.

[0088] Step 302: Acquire the data to be synchronized corresponding to the data synchronization task type information from the target database.

[0089] In some embodiments, the specific implementation of steps 301-302 and the technical effects thereof can be referred to in Figure 2 The corresponding steps 201-202 in the embodiments are not described in detail here.

[0090] Step 303 , in response to determining that the concurrent execution type is the first concurrent execution type, synchronizing each record in the data to be synchronized to the backup database in a serial manner.

[0091] In some embodiments, the execution body of the data synchronization method (eg Figure 1 The computing device 101 shown in the figure can synchronize each record in the data to be synchronized to the backup database in a serial manner via a wired connection or a wireless connection in response to determining that the concurrent execution type is the first concurrent execution type. The first concurrent execution type can be characterized by synchronizing the data to be synchronized in a serial manner.

[0092] Step 304, in response to determining that the concurrent execution type is the second concurrent execution type, generating at least one database operation statement based on each record in the data to be synchronized; and executing each database operation statement in the at least one database operation statement.

[0093] In some embodiments, the execution subject may generate at least one database operation statement based on each record in the data to be synchronized in response to determining that the concurrent execution type is the second concurrent execution type; and execute each database operation statement in the at least one database operation statement. The database operation statement may be an SQL statement for updating records in a database. The database operation statement may be an "UPDATE statement" in an SQL statement.

[0094] Optionally, the at least one database statement may include: at least one database insert statement. Optionally, the at least one database statement may also include: at least one sub-database operation statement. Among them, the database insert statement in the at least one database insert statement may be an SQL statement for adding records to a database. For example, the database insert statement may be an "INSERT" statement in an SQL statement. The second concurrent execution type may be characterized by synchronizing the data to be synchronized in a concurrent manner in units of data tables.

[0095] The execution subject may fill each data in each record in the to-be-synchronized data into a preset database operation statement to generate a database operation statement, thereby obtaining the at least one database operation statement.

[0096] First, the execution subject may add the at least one database operation statement to a target file, wherein the target file may be a file with a .sql suffix. Then, the execution subject may execute the command "source the name of the target file.sql" to execute each of the at least one database operation statement.

[0097] Optionally, in response to determining that the data to be synchronized includes the full amount of data, the execution subject may convert each record in the full amount of data into a database insert statement to obtain at least one database insert statement. The execution subject may concatenate each record in the full amount of data with the preset database operation statement to convert each record in the full amount of data into a database insert statement.

[0098] As an example, the record in the above data to be synchronized may be "0001, Zhang San, male, 23 years old". The above preset database operation statement may be "INSERT INTO backup database backup data table name VALUES()". The database insert statement corresponding to the above record "0001, Zhang San, male, 23 years old" may be "INSERT INTO backup database backup data table name VALUES(0001, Zhang San, male, 23 years old)".

[0099] Optionally, the execution subject generates at least one database operation statement based on each record in the data to be synchronized, which may include the following steps:

[0100] In the first step, in response to determining that the data to be synchronized includes incremental data, a candidate sub-database operation statement is generated according to an event type corresponding to each record in the incremental data to obtain at least one candidate sub-database operation statement.

[0101] The event type may represent an operation type. The event type may be an "update operation" type. The event type may also be an "insert operation" type. The event type may also be an "update operation" type.

[0102] As an example, the records included in the incremental data may include ["0002, Zhang Si, male, 22 years old", "0002, Zhang Wu, female, 21 years old", "0004, Zhang Liu, male, 24 years old", "0004, Zhang Liu, male, 23 years old"]. Among them, the event type corresponding to "0002, Zhang Si, male, 22 years old" may be "update operation type". Then the generated candidate sub-database operation statement may be "UPDATE data table 1 SET attribute 1 = "Zhang Si", attribute 2 = "male", attribute 3 = "22 years old" WHERE attribute 4 (primary key) = "0002"".

[0103] In the second step, in response to determining that the target data table meets the first parallel processing condition, the candidate sub-database operation statements that meet the merging condition in the at least one candidate sub-database operation statement are merged to generate a sub-database operation statement, thereby obtaining the at least one sub-database operation statement.

[0104] The target data table may be a data table corresponding to the data to be synchronized. The first parallel processing condition may be "the target data table includes a unique index or primary key". The merge condition may be "the primary key or unique index of the data table operated by the candidate sub-database operation statement is the same".

[0105] As an example, a candidate sub-database operation statement may be "UPDATE data table 1 SET attribute 1 = "Zhang Si", attribute 2 = "Male", attribute 3 = "22 years old" WHERE attribute 4 (primary key) = "0002"". The candidate sub-database operation statement may also be "UPDATE data table 1 SET attribute 1 = "Zhang Wu", attribute 2 = "Female", attribute 3 = "21 years old" WHERE attribute 4 (primary key) = "0002"". The exemplified candidate sub-database operation statements meet the merge condition. In response to determining that the target data table meets the first parallel processing condition, the above-mentioned execution entity may merge the exemplified candidate sub-database operation statements. The sub-database operation statement generated after the merge may be "UPDATE data table 1 SET attribute 1 = "Zhang Wu", attribute 2 = "Female", attribute 3 = "21 years old" WHERE attribute 4 (primary key) = "0002"".

[0106] As another example, the execution entity may merge the candidate sub-database operation statement "INSERT data table 2 INTO VALUES (0004, Zhang Liu, male, 24 years old)" with the candidate sub-database operation statement "UPDATE data table 2 SET attribute 1 = "Zhang Si", attribute 2 = "male", attribute 3 = "22 years old" WHERE attribute 4 (primary key) = "0002"". The sub-database operation statement generated after the merger may be "INSERT data table 2 INTO VALUES (0004, Zhang Liu, male, 23 years old)".

[0107] Optionally, in response to determining that the target data table satisfies the second parallel processing condition, each of the at least one database operation statement is executed in a serial working manner.

[0108] The second parallel processing condition is that “the target data table contains a unique index and a primary key.” The primary key may be a value used to uniquely identify a record in the data table.

[0109] As an example, a record may be “0001, Zhang San, Male”, where “0001” may be the primary key of the record.

[0110] Step 305 : In response to determining that the concurrent execution type is the third concurrent execution type, transaction parsing is performed on the data to be synchronized to generate at least one transaction to be processed.

[0111] In some embodiments, the execution subject may perform transaction parsing on the data to be synchronized in response to determining that the concurrent execution type is the third concurrent execution type, so as to generate at least one pending transaction. The execution subject may generate a pending transaction according to each transaction information of at least one transaction information included in the data to be synchronized, so as to obtain the at least one pending transaction. The pending transaction in the at least one pending transaction may be a database transaction. A database transaction may represent a sequence of database operations that access and may operate various data items. The third concurrent execution type may be characterized by data synchronization of the data to be synchronized with a transaction bit.

[0112] Step 306: determine the record information group corresponding to each pending transaction in at least one pending transaction to obtain a record information group set.

[0113] In some embodiments, the execution entity may determine at least one record operated by the transaction to be processed as a record information group corresponding to the transaction to be processed.

[0114] As an example, the SQL statement corresponding to the above pending transaction may be:

[0115] BEGIN

[0116] SELECT * FROM table name WHERE attribute 1 = attribute value 1

[0117] SELECT * FROM table name WHERE attribute 2 = attribute value 2

[0118] END

[0119] Step 307 : Select a predetermined number of pending transactions from at least one pending transaction as target pending transactions to obtain a target pending transaction set.

[0120] In some embodiments, the execution subject may select a predetermined number of pending transactions from at least one pending transaction as target pending transactions to obtain a target pending transaction set. Wherein, when the predetermined number is the same as the number of pending transactions in the at least one pending transaction, the execution subject may select a predetermined number of pending transactions from the at least one pending transaction as target pending transactions at a preset time point to obtain the target pending transaction set.

[0121] Step 308 , performing transaction conflict detection on the target pending transaction according to the record information group corresponding to each target pending transaction in the target pending transaction set, so as to generate transaction conflict detection information and obtain a transaction conflict detection information set.

[0122] In some embodiments, the execution subject performs transaction conflict detection on the target pending transaction according to the record information group corresponding to each target pending transaction in the target pending transaction set to generate transaction conflict detection information, and obtains the transaction conflict detection information set, which may include the following steps:

[0123] In the first step, a conflict detection algorithm is used to perform hash processing on each record information in the record information group corresponding to each target pending transaction in the target pending transaction set to generate a hash value to obtain a hash value group and a hash value group set.

[0124] The conflict detection algorithm may be an MD5 (MD5 Message-Digest) algorithm. The conflict detection algorithm may also be a CFD (conditional functional dependency) inconsistency detection algorithm.

[0125] As an example, the above record information may be “0001, Zhang San, male, 23 years old”, and the corresponding hash value may be “dd7019a6950caa2c9ddb2dfb62e3a445”.

[0126] In the second step, in response to determining that there are two hash values ​​with the same value in the above hash value set, transaction conflict detection information is generated according to the transaction identifier of the target transaction to be processed corresponding to each hash value in the above two hash values ​​to obtain the above transaction conflict detection information set.

[0127] Among them, the transaction conflict detection information may include: a first transaction identifier, a second transaction identifier and a transaction conflict identifier value. The first transaction identifier may be a transaction identifier corresponding to the first candidate pending transaction. The first candidate pending transaction may be a target pending transaction with a smaller timestamp among the target pending transactions corresponding to each of the two hash values. The second transaction identifier may be a transaction identifier corresponding to the second candidate pending transaction. The second candidate pending transaction may be a target pending transaction with a larger timestamp among the target pending transactions corresponding to each of the two hash values. The transaction conflict identifier value may be used to characterize whether there is a transaction conflict.

[0128] As an example, the transaction conflict detection information may be [a1, a2, 1]. Wherein, "a1" is the first transaction identifier. "a2" is the second transaction identifier. "1" is the transaction conflict identifier value. "1" may indicate that a transaction conflict exists.

[0129] Step 309: Generate a directed acyclic graph based on the target pending transaction set and the transaction conflict detection information set.

[0130] In some embodiments, the execution subject generates a directed acyclic graph based on the target pending transaction set and the transaction conflict detection information set, which may include the following steps:

[0131] The first step is to generate a target graph based on the above target pending transaction set.

[0132] The target graph may be a graph that only contains nodes but no edges. The nodes in the target graph correspond one-to-one to the target pending transactions in the target pending transaction set.

[0133] As an example, the target pending transaction set may be [a1, a2, a3, a4, a5]. Figure 4 shown. Figure 4 "a1", "a2", "a3", "a4", and "a5" in represent the respective nodes. Node "a1" corresponds to the target pending transaction "a1". Node "a2" corresponds to the target pending transaction "a2". Node "a3" corresponds to the target pending transaction "a3". Node "a4" corresponds to the target pending transaction "a4". Node "a5" corresponds to the target pending transaction "a5".

[0134] In the second step, according to each transaction conflict detection information in the transaction conflict detection information set, a directed edge is added to the target graph to generate the directed acyclic graph.

[0135] Among them, first, the execution subject may determine the node corresponding to the first transaction identifier included in the transaction conflict detection information as the source point. Then, the execution subject may determine the node corresponding to the second transaction identifier included in the transaction conflict detection information as the end point. Finally, the execution subject may add the directed edge from the source point to the end point to the target graph.

[0136] As an example, the transaction conflict detection information set may be {[a1, a4, 1], [a1, a2, 1], [a2, a5, 1], [a3, a4, 1]}. Figure 5 As shown. In the above directed acyclic graph, there is a directed edge between node "a1" and node "a4". There is a directed edge between node "a1" and node "a2". There is a directed edge between node "a2" and node "a5". There is a directed edge between node "a3" and node "a4".

[0137] Step 310 , topologically sort the directed acyclic graph to generate a topologically sorted sequence.

[0138] In some embodiments, the execution entity may perform topological sorting on the directed acyclic graph using a topological sorting algorithm to generate the topological sorting sequence.

[0139] The topological sorting algorithm may be a DFS (Depth-First-Search) algorithm or a Kahn algorithm.

[0140] As an example, the above topological sorting sequence can be Figure 6 As shown. Among them, Figure 6 The topological sorting sequence in includes three groups of nodes [(a1), (a2, a3), (a4, a5)].

[0141] Step 311: execute the target pending transaction corresponding to each node in the topological sorting sequence.

[0142] In some embodiments, the execution entity may execute the target pending transaction corresponding to each node in the topological sorting sequence.

[0143] like Figure 6As shown, the above execution subject can sequentially process the target pending transaction corresponding to the "(a1)" node in [(a1), (a2, a3), (a4, a5)] in a serial manner, the target processing transaction corresponding to the two nodes "(a2, a3)" and the target processing transaction corresponding to the two nodes "(a4, a5)". The target processing transactions corresponding to the two nodes "(a2, a3)" are executed concurrently. The target processing transactions corresponding to the two nodes "(a4, a5)" are executed concurrently.

[0144] from Figure 3 It can be seen that Figure 2Compared with the description of some corresponding embodiments, first, when the concurrent execution type is the first concurrent execution type, each record in the data to be synchronized is synchronized to the backup database in a serial manner. This ensures that the records can be written to the backup database one by one. Avoid the situation where data consistency may be destroyed due to concurrent backup. Secondly, when the concurrent execution type is the second concurrent execution type, each record in the data to be synchronized, including the full amount of data, is converted into a database insert statement. In actual situations, the full amount of data refers to the data that has been written into the data table. For such data, it can be backed up to the backup database through a database insert statement. For the data to be synchronized, including incremental data, such data is often generated after the data in the database is modified. Therefore, it is necessary to generate corresponding candidate sub-database operation statements according to the corresponding event type. In addition, concurrent synchronization often requires frequent read and write operations on the backup database, in order to avoid the read and write pressure of the backup database caused by frequent read and write. By merging the candidate sub-database operation statements that meet the merge conditions, a sub-database operation statement is generated. Thereby, the number of database operation statements is reduced, and then, the number of read and write times of the backup database is reduced, and the read and write pressure of the backup database is reduced. In addition, when the concurrent execution type is the third concurrent execution type, transaction parsing is performed on the data to be synchronized to generate at least one pending transaction. In actual situations, by generating a database operation statement and then executing the database operation statement, the execution cycle is relatively long, which reduces the data synchronization efficiency. Therefore, transaction parsing is performed on the data to be synchronized to generate at least one pending transaction. However, there may be transaction conflicts between different pending transactions, which may cause the database consistency to be destroyed. Therefore, transaction conflict detection is performed to determine the pending transactions with transaction conflicts. Then, a directed acyclic graph is generated based on the target pending transaction set and the transaction conflict detection information set. Then, the directed acyclic graph is topologically sorted to generate a topological sorting sequence. Finally, the target pending transactions corresponding to each node in the topological sorting sequence are executed. This ensures that the target pending transactions with transaction conflicts are synchronized in a serial manner, and the target pending transactions without transaction conflicts are synchronized in a concurrent manner. In this way, the flexibility of data synchronization is greatly improved, the data synchronization requirements in different scenarios are met, the data synchronization efficiency is improved, and the consumption of bandwidth resources is reduced.

[0145] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a data synchronization device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0146] like Figure 7As shown, the data synchronization device 700 of some embodiments includes: a receiving unit 701, an acquisition unit 702 and a synchronization unit 703. The receiving unit 701 is configured to receive data synchronization task type information in response to a successful connection with a target database; the acquisition unit 702 is configured to acquire the to-be-synchronized data corresponding to the data synchronization task type information from the target database; the synchronization unit 703 is configured to synchronize the to-be-synchronized data to a backup database according to a concurrent execution type, wherein the backup database is used to back up the data in the target database, and the concurrent execution type represents a method of synchronizing the to-be-synchronized data to the backup database.

[0147] In some optional implementations of some embodiments, the acquisition unit 702 may be further configured to: in response to determining that the data synchronization task type information is the first data synchronization type, acquire the full amount of data from the target database as the data to be synchronized.

[0148] In some optional implementations of some embodiments, the acquisition unit 702 may be further configured to: in response to determining that the data synchronization task type information is the second data synchronization type, acquire incremental data from the target database as data to be synchronized.

[0149] In some optional implementations of some embodiments, the acquisition unit 702 may be further configured to: in response to determining that the data synchronization task type information is the third data synchronization type, acquire full data and incremental data from the target database as the data to be synchronized.

[0150] In some optional implementations of some embodiments, the synchronization unit 703 may be further configured to: in response to determining that the concurrent execution type is the first concurrent execution type, synchronize each record in the data to be synchronized to the backup database in a serial manner.

[0151] In some optional implementations of some embodiments, the synchronization unit 703 can be further configured to: in response to determining that the above-mentioned concurrent execution type is the second concurrent execution type, generate at least one database operation statement based on each record in the above-mentioned data to be synchronized; and execute each database operation statement in the above-mentioned at least one database operation statement.

[0152] In some optional implementations of some embodiments, the at least one database statement may include: at least one database insert statement; and the synchronization unit 703 may be further configured to: in response to determining that the data to be synchronized includes the full amount of data, convert each record in the full amount of data into a database insert statement to obtain at least one database insert statement.

[0153] In some optional implementations of some embodiments, the at least one database statement may also include: at least one sub-database operation statement; and the synchronization unit 703 may be further configured to: in response to determining that the data to be synchronized includes incremental data, generate a candidate sub-database operation statement according to the event type corresponding to each record in the incremental data to obtain at least one candidate sub-database operation statement; in response to determining that the target data table satisfies the first parallel processing condition, merge the candidate sub-database operation statements that meet the merge condition in the at least one candidate sub-database operation statement to generate a sub-database operation statement to obtain the at least one sub-database operation statement, wherein the target data table is the data table corresponding to the data to be synchronized, the first parallel processing condition is that the target data table includes a unique index or a primary key, and the merge condition is that the primary key or unique index of the data table operated by the candidate sub-database operation statement is the same.

[0154] In some optional implementations of some embodiments, the synchronization unit 703 can be further configured to: in response to determining that the target data table satisfies the second parallel processing condition, execute each database operation statement of the at least one database operation statement in a serial working mode, wherein the second parallel processing condition is that the target data table contains a unique index and a primary key.

[0155] In some optional implementations of some embodiments, the synchronization unit 703 can be further configured to: in response to determining that the above-mentioned concurrent execution type is the third concurrent execution type, perform transaction parsing on the data to be synchronized to generate at least one pending transaction; determine the record information group corresponding to each pending transaction in the above-mentioned at least one pending transaction to obtain a record information group set; select a predetermined number of pending transactions from the above-mentioned at least one pending transaction as target pending transactions to obtain a target pending transaction set; perform transaction conflict detection on the above-mentioned target pending transactions according to the record information group corresponding to each target pending transaction in the above-mentioned target pending transaction set to generate transaction conflict detection information and obtain a transaction conflict detection information set; generate a directed acyclic graph based on the above-mentioned target pending transaction set and the above-mentioned transaction conflict detection information set, wherein the nodes in the above-mentioned directed acyclic graph correspond one-to-one to the target pending transactions in the above-mentioned target pending transaction set; perform topological sorting on the above-mentioned directed acyclic graph to generate a topological sorting sequence; and execute the target pending transactions corresponding to each node included in the above-mentioned topological sorting sequence.

[0156] Reference below Figure 8 , which shows an electronic device (such as Figure 1 A schematic diagram of the structure of the computing device 101)800 is shown. Figure 8The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0157] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0158] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 8 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0159] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0160] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0161] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0162] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: receives data synchronization task type information in response to successful connection with the target database; obtains the to-be-synchronized data corresponding to the data synchronization task type information from the target database; synchronizes the to-be-synchronized data to a backup database according to the concurrent execution type, wherein the backup database is used to back up the data in the target database, and the concurrent execution type represents the way to synchronize the to-be-synchronized data to the backup database.

[0163] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0164] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0165] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including a receiving unit, an acquisition unit, and a synchronization unit. The names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the receiving unit may also be described as "a unit for receiving data synchronization task type information when the connection with the target database is successful".

[0166] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0167] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A data synchronization method, comprising: In response to successfully connecting to the target database, receiving data synchronization task type information; Acquire the data to be synchronized corresponding to the data synchronization task type information from the target database; According to the concurrent execution type, the data to be synchronized is synchronized to a backup database, wherein the backup database is used to back up the data in the target database, and the concurrent execution type represents a method of synchronizing the data to be synchronized to the backup database, wherein the concurrent execution type includes: a second concurrent execution type, wherein the second concurrent execution type represents a type of data synchronization of the data to be synchronized in a concurrent manner in units of data tables, wherein: The step of synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: In response to determining that the concurrent execution type is the second concurrent execution type, generating at least one database operation statement based on each record in the to-be-synchronized data; executing each database operation statement in the at least one database operation statement, wherein the at least one database operation statement further includes: at least one sub-database operation statement; The step of generating at least one database operation statement based on each record in the data to be synchronized further includes: In response to determining that the data to be synchronized includes incremental data, generating candidate sub-database operation statements according to an event type corresponding to each record in the incremental data, to obtain at least one candidate sub-database operation statement; In response to determining that the target data table satisfies the first parallel processing condition, the candidate sub-database operation statements that meet the merge condition in the at least one candidate sub-database operation statement are merged to generate a sub-database operation statement, thereby obtaining the at least one sub-database operation statement, wherein the target data table is the data table corresponding to the data to be synchronized, the first parallel processing condition is that the target data table includes a unique index or a primary key, and the merge condition is that the primary key or unique index of the data table operated by the candidate sub-database operation statements is the same.

2. The method according to claim 1, wherein: The acquiring the to-be-synchronized data corresponding to the data synchronization task type information from the target database includes: In response to determining that the data synchronization task type information is the first data synchronization type, full data is acquired from the target database as the data to be synchronized.

3. The method according to claim 1, wherein: The acquiring the to-be-synchronized data corresponding to the data synchronization task type information from the target database includes: In response to determining that the data synchronization task type information is the second data synchronization type, incremental data is acquired from the target database as data to be synchronized.

4. The method according to claim 1, wherein: The acquiring the to-be-synchronized data corresponding to the data synchronization task type information from the target database includes: In response to determining that the data synchronization task type information is the third data synchronization type, full data and incremental data are acquired from the target database as the data to be synchronized.

5. The method according to claim 1, wherein: The step of synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: In response to determining that the concurrent execution type is the first concurrent execution type, each record in the to-be-synchronized data is synchronized to the backup database in a serial manner.

6. The method according to any one of claims 2 to 4, wherein: The at least one database operation statement includes: at least one database insert statement; and The generating at least one database operation statement based on each record in the data to be synchronized includes: In response to determining that the data to be synchronized includes full data, each record in the full data is converted into a database insert statement to obtain at least one database insert statement.

7. The method according to claim 6, wherein: The executing each database operation statement of the at least one database operation statement comprises: In response to determining that the target data table satisfies a second parallel processing condition, each database operation statement in the at least one database operation statement is executed in a serial working manner, wherein the second parallel processing condition is that the target data table contains a unique index and a primary key.

8. The method according to any one of claims 2 to 4, wherein: The step of synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: In response to determining that the concurrent execution type is a third concurrent execution type, performing transaction parsing on the data to be synchronized to generate at least one transaction to be processed; Determine a record information group corresponding to each pending transaction in the at least one pending transaction to obtain a record information group set; Selecting a predetermined number of pending transactions from the at least one pending transaction as target pending transactions to obtain a target pending transaction set; According to the record information group corresponding to each target to-be-processed transaction in the target to-be-processed transaction set, transaction conflict detection is performed on the target to-be-processed transaction to generate transaction conflict detection information, thereby obtaining a transaction conflict detection information set; Based on the target pending transaction set and the transaction conflict detection information set, a directed acyclic graph is generated, wherein the nodes in the directed acyclic graph correspond one-to-one to the target pending transactions in the target pending transaction set; Performing topological sorting on the directed acyclic graph to generate a topological sorting sequence; Execute the target pending transaction corresponding to each node in the topological sorting sequence.

9. A data synchronization device, comprising: A receiving unit, configured to receive data synchronization task type information in response to a successful connection with a target database; an acquisition unit, configured to acquire, from the target database, the data to be synchronized corresponding to the data synchronization task type information; The synchronization unit is configured to synchronize the data to be synchronized to a backup database according to a concurrent execution type, wherein the backup database is used to back up the data in the target database, and the concurrent execution type represents a method of synchronizing the data to be synchronized to the backup database, wherein the concurrent execution type includes: a second concurrent execution type, wherein the second concurrent execution type represents a type of data synchronization of the data to be synchronized in a concurrent manner in units of data tables, wherein: The step of synchronizing the data to be synchronized to the backup database according to the concurrent execution type includes: In response to determining that the concurrent execution type is the second concurrent execution type, generating at least one database operation statement based on each record in the to-be-synchronized data; executing each database operation statement in the at least one database operation statement, wherein the at least one database operation statement further includes: at least one sub-database operation statement; The step of generating at least one database operation statement based on each record in the data to be synchronized further includes: In response to determining that the data to be synchronized includes incremental data, generating candidate sub-database operation statements according to an event type corresponding to each record in the incremental data, to obtain at least one candidate sub-database operation statement; In response to determining that the target data table satisfies the first parallel processing condition, the candidate sub-database operation statements that meet the merge condition in the at least one candidate sub-database operation statement are merged to generate a sub-database operation statement, thereby obtaining the at least one sub-database operation statement, wherein the target data table is the data table corresponding to the data to be synchronized, the first parallel processing condition is that the target data table includes a unique index or a primary key, and the merge condition is that the primary key or unique index of the data table operated by the candidate sub-database operation statements is the same.

10. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images