Database-based data synchronization method, apparatus, medium, device and product

Abstract

The present disclosure relates to a database-based data synchronization method, device, medium, equipment and product, and in particular to the technical field of data transmission, which writes the current data synchronization point into the log file of the database through the target statement of the data definition language type carrying the current data synchronization point, and finds the target data synchronization point in the log file of the database when data breakpoint transmission, and then performs data synchronization based on the target data synchronization point, which can play the role of data fence, avoid the invalidation of the data synchronization point in the scene of master-slave database switching, so that the consistency and integrity of the data can be guaranteed in the data synchronization process in the scene of master-slave database switching. Moreover, the data synchronization point can also be supported by various versions of MySQL database through the target statement of the data definition language.

Description

Technical Field

[0001] This disclosure relates to the field of data transmission technology, and more specifically, to a database-based data synchronization method, apparatus, medium, device, and product. Background Technology

[0002] During data synchronization, it is necessary to continuously record the current synchronization point information. When data synchronization is interrupted due to network problems, system failures, or other reasons, the system needs to be able to detect this situation and save the current point information. After the interruption problem is resolved, the system will use the previously saved point information to resume data synchronization. Therefore, recording point information is crucial for resuming interrupted data transmission.

[0003] In related technologies, MySQL (a relational database management system) uses two types of data synchronization points: physical points based on binary filenames and offsets, and logical points based on GTID / GTIDSet. However, in master-slave failover scenarios, the physical points based on binary filenames and offsets become invalid because the master and slave databases are physically located different. While logical points based on GTID / GTIDSet can solve the problem of point failure during master-slave failover, older versions of MySQL do not support their use. Therefore, there is an urgent need for a new data synchronization method that can adapt to various versions of MySQL and solve the problem of point failure during master-slave failover. Summary of the Invention

[0004] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] Firstly, this disclosure provides a database-based data synchronization method, including:

[0006] Execute a target statement carrying the current data synchronization point in the database to write the current data synchronization point to the database log file through the target statement, wherein the target statement is a statement of the Data Definition Language type;

[0007] When resuming data transmission from a breakpoint, the target data synchronization point is found in the log file of the database. The target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted.

[0008] Data synchronization is performed based on the target data synchronization point.

[0009] Secondly, this disclosure provides a database-based data synchronization device, comprising:

[0010] The execution module is configured to execute a target statement carrying the current data synchronization point in the database, so as to write the current data synchronization point to the database log file through the target statement, wherein the target statement is a statement of data definition language type;

[0011] The lookup module is configured to find the target data synchronization point in the log file of the database when resuming data transmission from a breakpoint. The target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted.

[0012] The synchronization module is configured to perform data synchronization based on the target data synchronization point.

[0013] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0014] Fourthly, this disclosure provides an electronic device, comprising:

[0015] A storage device on which computer programs are stored;

[0016] A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0017] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0018] Based on the above technical solution, by executing a target statement of type Data Definition Language (DDL) carrying the current data synchronization point in the database, the target statement writes the current data synchronization point to the database log file. During data resumption from a breakpoint, the target data synchronization point is located in the database log file, and data synchronization is performed based on this target data synchronization point. This acts as a data barrier, preventing the data synchronization point from becoming invalid during master-slave database switching scenarios. Therefore, data consistency and integrity are guaranteed during data synchronization in master-slave database switching scenarios. Furthermore, using DDL target statements to embed data synchronization points also supports various versions of MySQL databases.

[0019] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] Figure 1 This is a flowchart illustrating a database-based data synchronization method according to some embodiments.

[0022] Figure 2 yes Figure 1 The detailed flowchart of step 120 is shown.

[0023] Figure 3 This is a schematic diagram of a database-based data synchronization device according to some embodiments.

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device according to some embodiments. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0031] Figure 1 This is a flowchart illustrating a database-based data synchronization method according to some embodiments. For example... Figure 1 As shown, this disclosure provides a database-based data synchronization method. This method can be executed by an electronic device, specifically by a database-based data synchronization device. This device can be implemented in software and / or hardware and configured within the electronic device. It should be understood that the electronic device can be a server. Figure 1 As shown, the method may include the following steps.

[0032] In step 110, a target statement carrying the current data synchronization point is executed in the database to write the current data synchronization point to the database log file. The target statement is a statement of the Data Definition Language type.

[0033] Here, the database can be a source database, which refers to the database at the starting point of the data. The source database is where the original data resides. Data is read from the source database and copied or synchronized to another database (the target database). A data synchronization point is a marker recording the current synchronization status during the data synchronization process. The data synchronization point can consist of a synchronization timestamp or the log position of data changes (such as the binlog filename and offset). The current data synchronization point refers to the real-time position information of the current data synchronization process.

[0034] The target statement is a Data Definition Language (DDL) statement. DDL statements are a type of Structured Query Language (SQL) used to define and manage the structure of a database. They are typically used to create, modify, and delete database objects such as databases, tables, views, indexes, triggers, and stored procedures.

[0035] During data synchronization, the Data Transmission Service (DTS) periodically executes target statements carrying the current data synchronization point in the database. These target statements are recorded in the database's log file, thus recording the current data synchronization point carried by the target statements. The log file can be a binary log (binlog) file.

[0036] It's worth noting that due to the MTS (Multi-Threaded Slave) mechanism of master-slave replication, DML (Data Manipulation Language) statements are not strictly ordered between the master and slave databases. For example, if insert1, insert2, and insert3 are executed sequentially on the source database, since these three statements do not conflict, the execution order on the slave database might be insert3, insert2, and insert1. Therefore, DML statements between master and slave databases are not ordered. Consequently, if DML statements are used to embed the current data synchronization point in the log file, this synchronization point will become invalid during a master-slave database switchover.

[0037] In this embodiment, the current data synchronization point is written to the log file using a target statement of the Data Definition Language (DDL) type. Because DDL statements have a checkpoint mechanism, their order is not out of order. In a scenario where the master database fails and data synchronization is performed by the slave database, the slave database waits for the transactions preceding the DDL statement to complete before executing the DDL statement. After the DDL statement is executed, subsequent statements are allowed to proceed. Therefore, writing the current data synchronization point to the log file using a target statement of the DDL type acts as a data barrier, ensuring data consistency and integrity during data synchronization in a master-slave database switchover scenario.

[0038] Furthermore, since DDL statements are a type of SQL statement, all versions of MySQL databases can use the target statements provided in this disclosure to mark data synchronization points. For example, MySQL databases versions 5.5 and earlier can use the database-based data synchronization method provided in this disclosure to record data synchronization points to support breakpoint resumption in master-slave switching scenarios.

[0039] It is worth noting that the data synchronization point in this embodiment is a logical point. A logical point refers to a point defined based on the logical order of data changes.

[0040] In step 120, when resuming data transmission from a breakpoint, the target data synchronization point is found in the database log file. The target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted.

[0041] Here, data resume refers to the ability to continue transmitting unfinished data from the point of interruption during data synchronization if the transmission is interrupted due to network problems or other reasons, without having to restart the entire transmission process. Through step 110 above, the current data synchronization point can be continuously recorded in the database log file during the data synchronization process. When data synchronization is interrupted due to network problems, system failures, or other reasons, the current data synchronization point corresponding to the moment of interruption will also be recorded in the log file.

[0042] After resolving the issue causing the data synchronization interruption, data transmission is resumed from the point of interruption via the data transmission service. During the resumption, the target data synchronization point is retrieved from the database log file using the data transmission service. This target data synchronization point is the current data synchronization point recorded at the time of the last data synchronization interruption. For example, the target data synchronization point can be understood as the last data synchronization point recorded in the log file.

[0043] In some embodiments, the target statement can be found in the database's log file, and then the target data synchronization point can be determined using the found target statement.

[0044] In step 130, data synchronization is performed based on the target data synchronization point.

[0045] Here, data synchronization can be performed based on a target data synchronization point using a data transfer service. This target data synchronization point includes the binlig filename and offset from the point where data synchronization was last interrupted. When resuming interrupted data transfer, the data is read again and synchronized with the target database starting from the last interrupted binlig file and offset.

[0046] Therefore, by executing a target statement of type Data Definition Language (DDL) carrying the current data synchronization point in the database, and writing the current data synchronization point to the database log file, and then searching for the target data synchronization point in the database log file during data resumption from a breakpoint, and performing data synchronization based on the target data synchronization point, a data barrier effect can be achieved. This prevents the data synchronization point from becoming invalid in master-slave database switching scenarios, ensuring data consistency and integrity during data synchronization in master-slave database switching scenarios. Moreover, using DDL target statements to embed data synchronization points also supports various versions of MySQL databases.

[0047] In some feasible implementations, the target statement includes the current data synchronization point and the target table name. The table corresponding to the target table name does not belong to the database, and the target statement is used to instruct the deletion of the corresponding table based on the target table name.

[0048] For example, the target statement could be `DROP TABLE IF EXISTS`. It should be understood that `DROP TABLE IF EXISTS` is an SQL statement used to delete a table from a database. `DROP TABLE IF EXISTS` is typically followed by `table_name`, which indicates the name of the table to be deleted. Therefore, the current data synchronization point and the target table name can be carried in the `table_name` position.

[0049] For example, the target statement could be:

[0050] DROP TABLE IF EXISTS ${NOT_EXISTS_TABLE_PREFIX}_${DATA}.

[0051] Here, NOT_EXISTS_TABLE_PREFIX represents the name of the target table that does not belong to the database, and DATA represents the current data synchronization point.

[0052] It's worth noting that "the table corresponding to the target table name does not belong to the database" means that the table corresponding to the target table name does not exist in the database. The target table name included in the target statement can refer to the full name of the target table or a prefix of the target table name.

[0053] Accordingly, in step 110, the target statement can be executed in the database to delete the table corresponding to the target table name.

[0054] Here, when the target statement is executed, since the table corresponding to the target table name does not exist in the database, the database will not report an error but will silently ignore the deletion operation of the target statement. Therefore, the target statement can be recorded in the log file. Moreover, by using the target statement to delete a table in the database, the user will not be aware of the operation of deleting a table corresponding to a non-existent target table name. Therefore, writing the current data synchronization point to the log file through the target statement is imperceptible to the user.

[0055] Therefore, the aforementioned target statement can write the data synchronization point to the database's log file. Furthermore, the action of writing the data synchronization point to the log file is imperceptible to the user, avoiding interference and improving the user experience.

[0056] In some feasible implementations, the target statement may include the current data synchronization point and a specified table name, and the target statement is used to instruct the creation of a corresponding table based on the specified table name.

[0057] For example, the target statement could be `CREATE TABLE IF NOT EXISTS`. It should be understood that `CREATE TABLE IF NOT EXISTS` is an SQL statement used to create a new table in the database. `CREATE TABLE IFNOT EXISTS` is typically followed by `table_name`, which indicates the name of the table to be created. Therefore, the current data synchronization point and the specified table name can be carried in the `table_name` position.

[0058] The specified table name can be a user-defined table name specifically used to record data synchronization points. For example, the target statement could be: `CREATE TABLE IF NOT EXISTS ${TABLE_A}_${DATA}`, where `TABLE_A` is the specified table name and `DATA` represents the current data synchronization point.

[0059] It is worth noting that the table name specified in the target statement can be different each time it is executed. The specified table name included in the target statement can refer to the full table name or a prefix of the specified table name.

[0060] Accordingly, in step 110, the target statement can be executed in the database to create a table corresponding to the specified table name.

[0061] Here, when the target statement is executed in the database, if the specified table name does not exist, the table corresponding to the specified table name will be created in the database. Therefore, the target statement can be recorded in the log file.

[0062] It should be understood that, compared to DROP TABLE IF NOT EXISTS, CREATE TABLE IF NOT EXISTS requires the creation of a table in the database, so the operation is noticeable to the user, while DROP TABLE IF EXISTS is imperceptible to the user.

[0063] It is worth noting that in this embodiment, the current data synchronization point can be an incrementing sequence number. Of course, the current data synchronization point can also be a timestamp. In practical applications, the data synchronization point can be defined according to the actual situation, but it must be ensured that the data synchronization points have a partial order relationship.

[0064] Therefore, by using the above target statement, the data synchronization point can be written to the database log file by creating a table in the database.

[0065] In some feasible implementations, in step 110, the current data synchronization point during the data synchronization process can be periodically obtained, and a target statement can be constructed based on a preset table name and the current data synchronization point. The target statement is then executed in the database to write the target statement into the database's log file.

[0066] Here, during data synchronization, the current data synchronization point can be periodically obtained through the data transmission service, and the current data point can be recorded through the target statement. The interval between each period can be set according to the actual situation.

[0067] The preset table name can be the target table name described in the above embodiments or a specified table name. The target statement can be constructed by carrying the preset table name and the current data synchronization point in DROP TABLE IF EXISTS or CREATE TABLE IF NOT EXISTS.

[0068] Then, the target statement is executed in the database via the data transfer service. It's important to note that the executed target statement is recorded in the database's log file, thus writing the data synchronization point carried in the target statement to the database's log file.

[0069] For example, a preset table name and the current data synchronization point can be written into the corresponding DDL statement to construct the target statement. The target statement can be either DROP TABLE IF EXISTS or CREATE TABLE IF NOT EXISTS as described in the above embodiments.

[0070] It should be understood that during the data synchronization process, the data transmission service periodically obtains the current data synchronization point, constructs the target statement based on the current data synchronization point and the preset table name, and then executes the target statement in the database.

[0071] Therefore, through the above implementation method, the target statement can be executed periodically to write the data synchronization point during the data synchronization process into the log file.

[0072] Figure 2 yes Figure 1 The detailed flowchart for step 120 is shown below. Figure 2 As shown, in some possible implementations, step 120 includes the following steps.

[0073] In step 121, each log file in the database is scanned to obtain the first data synchronization point recorded in each log file.

[0074] In the above implementation, the data transmission service periodically executes the target statement and writes the data synchronization point to the database's log file. During breakpoint resumption, the data transmission service scans each log file of the database to obtain the first data synchronization point recorded in each log file.

[0075] It should be noted that data synchronization points have a partial order relationship. For example, data synchronization points can be incremental sequence numbers or timestamps. Therefore, by scanning the first data synchronization point recorded in each log file, the log file containing the target data synchronization point can be determined.

[0076] In some embodiments, the target statement also includes a preset table name. Accordingly, based on the preset table name, the first target statement recorded in each log file can be found in each log file of the database, and the first data synchronization point recorded in each log file can be determined based on the data synchronization point carried in the found target statement.

[0077] The preset table name can be the target table name described in the above embodiments or a specified table name.

[0078] In this embodiment of the disclosure, the data synchronization point is recorded in the target statement of the log file. Therefore, the target statement used to record the data synchronization point can be found in each log file of the database by using a preset table name. Due to the partial order relationship of the data synchronization points, the first target statement found in the log file is the first data synchronization point recorded in that log file.

[0079] After finding the target statement in each log file, the first data synchronization point recorded in that log file can be determined by the data synchronization point carried in the first target statement found in each log file.

[0080] It should be understood that the target statement used to record data synchronization points can be quickly found in the log file by using the preset table name.

[0081] In step 122, the target log file is determined from each log file according to the order corresponding to each first data synchronization point.

[0082] Here, because the data synchronization points have a partial order relationship, the target log file can be determined from each log file according to the order corresponding to each first data synchronization point. The target log file is the log file containing the target data synchronization point used for resuming interrupted downloads.

[0083] For example, the log file corresponding to the first data synchronization point with the highest order can be determined as the target log file.

[0084] For example, assuming the first data synchronization points detected by scanning are 1, 11, 21, and 31, then the log file containing the data synchronization point corresponding to 31 is the target log file.

[0085] In step 123, the target data synchronization point is determined from each data synchronization point according to the order of the data synchronization points recorded in the target log file.

[0086] Here, based on the preset table name, all target statements can be found in the target log file, and the synchronization points of each data recorded in the target log file can be determined according to all target statements.

[0087] For example, since the data synchronization sites have a partial order relationship, the data synchronization site with the largest order among the data synchronization sites recorded in the target log file is determined as the target data synchronization site.

[0088] Therefore, by implementing the above method, the target data synchronization point can be determined by scanning the first data synchronization point recorded in the log file, without having to scan all points in the log file. This not only improves the scanning speed but also saves computing resources.

[0089] In some feasible implementations, in step 110, the first target statement recorded in each log file of the database can be found based on the preset table name, and then the target data synchronization point can be determined based on the data synchronization point carried in the found target statement.

[0090] Here, the table name can be matched with the statements recorded in the log file to find the first target statement recorded in each log file.

[0091] Due to the partial order of data synchronization points, the first target statement found in the log file is the first data synchronization point that records the log file.

[0092] At this point, the target log file can be determined from each log file based on the order of the first data synchronization points. Then, the target data synchronization point can be determined from each data synchronization point according to the order of the data synchronization points recorded in the target log file.

[0093] Therefore, through the above implementation method, the target data synchronization point can be quickly and accurately determined in the log file by searching for the preset table name.

[0094] It is worth noting that the database-based data synchronization method provided in this disclosure, by periodically executing target statements of the Data Definition Language (DDL) type during the data synchronization process, not only provides a solution for recording logical positions for older versions of MySQL databases, but also supports data interruption resumption in master-slave database switching scenarios through the DDL statement checkpoint mechanism, ensuring data consistency and integrity. Furthermore, the database-based data synchronization method provided in this disclosure is also applicable to master-slave replication scenarios. In scenarios where transaction identifiers (IDs) are not inherited, data synchronization positions can also be recorded based on the target statements of the DDL type provided in this disclosure. Moreover, when the target statement of the DDL type is a DROP TABLE IF EXISTS statement, it is imperceptible to the user, greatly improving the user experience.

[0095] Figure 3 This is a schematic diagram illustrating the structure of a database-based data synchronization device according to some embodiments. For example... Figure 3 As shown, this disclosure provides a database-based data synchronization device 300, which includes:

[0096] The execution module 301 is configured to execute a target statement carrying the current data synchronization point in the database, so as to write the current data synchronization point into the log file of the database through the target statement, wherein the target statement is a statement of the Data Definition Language type;

[0097] The lookup module 302 is configured to find the target data synchronization point in the log file of the database when resuming data transmission from a breakpoint. The target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted.

[0098] Synchronization module 303 is configured to perform data synchronization based on the target data synchronization point.

[0099] Optionally, the target statement includes the current data synchronization point and the target table name, wherein the table corresponding to the target table name does not belong to the database, and the target statement is used to instruct the deletion of the corresponding table based on the target table name;

[0100] The execution module 301 is specifically configured as follows:

[0101] Execute the target statement in the database to delete the table corresponding to the target table name.

[0102] Optionally, the target statement includes the current data synchronization point and a specified table name, and the target statement is used to instruct the creation of a corresponding table based on the specified table name;

[0103] The execution module 301 is specifically configured as follows:

[0104] The target statement is executed in the database to create a table corresponding to the specified table name in the database.

[0105] Optionally, the execution module 301 is specifically configured as follows:

[0106] Periodically retrieve the current data synchronization point during the data synchronization process;

[0107] Based on the preset table name and the current data synchronization point, construct the target statement;

[0108] The target statement is executed in the database to write the target statement to the database's log file.

[0109] Optionally, the search module 302 is specifically configured as follows:

[0110] Scan each log file in the database to obtain the first data synchronization point recorded in each log file;

[0111] The target log file is determined from each of the log files according to the order corresponding to the first data synchronization point.

[0112] The target data synchronization point is determined from among the data synchronization points recorded in the target log file according to their order.

[0113] Optionally, the target statement also includes a preset table name; the search module 302 is specifically configured as follows:

[0114] The step of scanning each log file of the database to obtain the first data synchronization point recorded in each log file includes:

[0115] Based on the preset table name, the first target statement recorded in each log file of the database is found.

[0116] The target data synchronization point is determined based on the data synchronization point carried in the target statement.

[0117] The logic of the methods executed by each functional module in the database-based data synchronization device 300 can be referred to the relevant method section of the above embodiment, and will not be repeated here.

[0118] The following is for reference. Figure 4 The diagram illustrates a structural schematic suitable for implementing an electronic device 400 according to embodiments of the present disclosure. In embodiments of the present disclosure, the electronic device may be a server, for example, a server hosting a database. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0119] like Figure 4 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. Processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0120] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0121] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0122] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or synchronize a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be synchronized with any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some implementations, the server may communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and may interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0124] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0125] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: execute a target statement carrying the current data synchronization point in a database, thereby writing the current data synchronization point into the database's log file via the target statement, wherein the target statement is a statement of the Data Definition Language type; when resuming data transmission from a breakpoint, locate the target data synchronization point in the database's log file, wherein the target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted; and perform data synchronization based on the target data synchronization point.

[0126] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0129] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0130] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0132] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0133] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A database-based data synchronization method, characterized in that, include: Execute a target statement carrying the current data synchronization point in the database to write the current data synchronization point to the database's log file. The target statement is a Data Definition Language (DDL) statement; the target statement is recorded in the log file; the target statement includes a preset table name and the current data synchronization point. When resuming data transmission from a breakpoint, the target statement is found in the database log file according to the preset table name. The target data synchronization point is determined by the found target statement. The target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted. Data synchronization is performed based on the target data synchronization point; The target statement includes the current data synchronization point and the target table name. The table corresponding to the target table name does not belong to the database. The target statement is used to instruct the deletion of the corresponding table based on the target table name. Executing the target statement carrying the current data synchronization point in the database includes: Execute the target statement in the database to delete the table corresponding to the target table name; or; The target statement includes the current data synchronization point and a specified table name. The target statement is used to instruct the creation of a corresponding table based on the specified table name. Executing the target statement carrying the current data synchronization point in the database includes: The target statement is executed in the database to create a table corresponding to the specified table name in the database.

2. The method according to claim 1, characterized in that, The execution of the target statement carrying the current data synchronization point in the database includes: Periodically retrieve the current data synchronization point during the data synchronization process; Based on the preset table name and the current data synchronization point, construct the target statement; The target statement is executed in the database to write the target statement to the database's log file.

3. The method according to claim 1 or 2, characterized in that, The target data synchronization point is obtained through the following steps: Scan each log file of the database to obtain the first data synchronization point recorded in each log file. The first data synchronization point is the data synchronization point included in the first target statement found in the log file based on the preset table name. The target log file is determined from each of the log files according to the order corresponding to the first data synchronization point. The target data synchronization point is determined from among the data synchronization points recorded in the target log file according to their order.

4. The method according to claim 1 or 2, characterized in that, The target data synchronization point is obtained through the following steps: Based on the preset table name, the first target statement recorded in each log file of the database is found. The target data synchronization point is determined based on the data synchronization point carried in the target statement.

5. A database-based data synchronization device, characterized in that, include: The execution module is configured to execute a target statement carrying the current data synchronization point in the database, so as to write the current data synchronization point to the database log file through the target statement, wherein the target statement is a statement of data definition language type; the target statement is recorded in the log file; the target statement includes a preset table name and the current data synchronization point; The lookup module is configured to, when resuming data transmission from a breakpoint, search for the target statement in the database log file according to the preset table name, and determine the target data synchronization point by the found target statement. The target data synchronization point is the current data synchronization point recorded when the last data synchronization was interrupted. The synchronization module is configured to perform data synchronization based on the target data synchronization point; The target statement includes the current data synchronization point and the target table name. The table corresponding to the target table name does not belong to the database. The target statement is used to instruct the deletion of the corresponding table based on the target table name. The execution module is specifically configured to execute the target statement in the database and delete the table corresponding to the target table name. or; The target statement includes the current data synchronization point and a specified table name. The target statement is used to instruct the creation of a corresponding table based on the specified table name. The execution module is specifically configured to execute the target statement in the database and create a table corresponding to the specified table name in the database.

6. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processing device, it implements the steps of the method according to any one of claims 1-4.

7. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images