Data synchronization method, electronic device, storage medium and computer program product

CN122507802APending Publication Date: 2026-08-04BAIRONG ZHIXIN (BEIJING) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
BAIRONG ZHIXIN (BEIJING) TECH CO LTD
Filing Date
2026-05-26
Publication Date
2026-08-04

AI Technical Summary

Technical Problem

[0003]然而,相关的技术中,通常依赖较多的外部组件、通过复杂的架构实现数据同步,导致配置复杂、可用性差、运维成本高

Benefits of technology

[0022]上述技术方案中,每个脚本程序获取第一数据库中的至少一个待同步数据表,并获取每个待同步数据表在上一次同步操作对应的同步信息,利用备脚本程序,针对在上一次同步操作对应的第一结束时间与当前时间的差值大于第一时间阈值的第一待同步数据表执行同步操作,并利用主脚本程序,针对第二待同步数据表执行同步操作,以将第一待同步数据表和第二待同步数据表同步至第二数据库。由此,利用脚本程序实现数据同步,无需依赖外部组件,具有轻量、灵活、易于配置和维护的优点,并且备脚本程序可以对滞后未同步的数据进行补偿同步,或在主脚本程序出现故障时,备脚本程序可以接管其工作,确保同步任务不中断,可以保证数据同步的实时性和可靠性。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122507802A_ABST
    Figure CN122507802A_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a data synchronization method, an electronic device, a storage medium and a computer program product. The data synchronization method comprises: obtaining, by using a script program, a to-be-synchronized data table in a first database, and obtaining an end time corresponding to a last synchronization operation of each to-be-synchronized data table, executing, by using a backup script program, a synchronization operation on a first to-be-synchronized data table (a lagging to-be-synchronized data table), and executing, by using a main script program, a synchronization operation on a second to-be-synchronized data table (a normal to-be-synchronized data table), to synchronize the first to-be-synchronized data table and the second to-be-synchronized data table to a second database. The script program does not need to depend on external components, is lightweight, flexible, easy to configure and maintain, and the backup script program can perform compensation synchronization on the lagging unsynchronized data, and when the main script program fails, the backup script program can take over its work, and the real-time performance and reliability of data synchronization can be ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, it relates to a data synchronization method, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of science and technology, transactional processing and analytical processing are often handled by different types of databases. Transactional databases can efficiently handle the writing of high-concurrency, large-scale business data; while analytical databases can meet the needs of low-latency detailed queries and aggregate analysis in complex analytical scenarios. Typically, data from transactional databases needs to be synchronized to analytical databases to combine the technical advantages of both and meet the requirements of efficient data analysis.

[0003] However, the relevant technologies usually rely on many external components and achieve data synchronization through complex architectures, resulting in complex configurations, poor availability, and high operation and maintenance costs. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems.

[0005] According to one aspect of the present invention, a data synchronization method is provided. The data synchronization method includes:

[0006] Using each of the multiple script programs, at least one data table to be synchronized in the first database is obtained, and for each of the at least one data table to be synchronized, the synchronization information corresponding to the data table to be synchronized in the previous synchronization operation is obtained. The multiple script programs include a main script program and a backup script program, and the synchronization information includes the end time of the data synchronized in the corresponding synchronization operation.

[0007] Using a backup script program, a synchronization operation is performed on the first data table to be synchronized, and using a main script program, a synchronization operation is performed on the second data table to be synchronized, so as to synchronize the first data table to be synchronized and the second data table to be synchronized to the second database.

[0008] The first data table to be synchronized is the data table whose difference between the first end time corresponding to the last synchronization operation and the current time is greater than the first time threshold. The second data table to be synchronized is at least one data table to be synchronized other than the first data table to be synchronized.

[0009] For example, the synchronization information also includes the start time of the data synchronized in the corresponding synchronization operation; the synchronization operation includes: determining the second start time and the second end time corresponding to the current synchronization operation based on the first start time or the first end time corresponding to the previous synchronization operation of the data table to be synchronized; and synchronizing the data in the data table to be synchronized whose update time is between the second start time and the second end time to the second database.

[0010] For example, synchronizing data in the data table to be synchronized whose update time is between the second start time and the second end time to the second database includes: establishing a connection between the second database and the first database using an external directory service; querying the data in the data table to be synchronized whose update time is between the second start time and the second end time using the external directory service in the first database, and writing the queried data into the second database.

[0011] For example, the synchronization information also includes the synchronization status of the corresponding synchronization operation; determining the second start time and the second end time corresponding to the current synchronization operation based on the first start time or the first end time of the data table to be synchronized in the previous synchronization operation includes: determining the second start time based on the first end time when the synchronization status indicates that the synchronization is successful; determining the second start time based on the first start time when the synchronization status indicates that the synchronization has failed; and determining the second end time based on the second start time and the first preset duration after determining the second start time.

[0012] For example, a backup script program is used to perform a synchronization operation on a first data table to be synchronized, and a main script program is used to perform a synchronization operation on a second data table to be synchronized. The method includes: traversing each of at least one data table to be synchronized, and for the currently traversed data table to be synchronized, when the synchronization status of the data table to be synchronized indicates that the synchronization is successful or failed, performing a synchronization operation using the backup script program or the main script program; the method further includes: when the synchronization status indicates that the synchronization is in progress, starting to traverse the next data table to be synchronized.

[0013] For example, the method further includes: after the step of synchronizing data in the data table to be synchronized whose update time is between the second start time and the second end time to the second database begins to be executed, starting to traverse the next data table to be synchronized.

[0014] For example, traversing each of the at least one data tables to be synchronized includes: traversing each of the at least one data tables to be synchronized according to a preset dependency order.

[0015] For example, determining the second end time based on the second start time and the first preset duration includes: if the first reference time is later than or equal to the second reference time, then the second end time is determined as the second reference time; if the first reference time is earlier than the second reference time, then the second end time is determined as the first reference time; wherein the first reference time is the sum of the second start time and the first preset duration, and the second reference time is the difference between the current time and the second preset duration.

[0016] For example, obtaining the synchronization information corresponding to the previous synchronization operation for the data table to be synchronized includes: querying the synchronization record table based on the table name of the data table to be synchronized; if the synchronization information corresponding to the data table to be synchronized is found, the latest synchronization information found is determined as the synchronization information corresponding to the previous synchronization operation; if the synchronization information corresponding to the data table to be synchronized is not found, an initial synchronization information record is added to the synchronization record table, wherein the synchronization information also includes the table name of the data table to be synchronized, and the end time in the initial synchronization information is determined based on the current time; the synchronization operation also includes: adding a synchronization information record corresponding to the current synchronization operation to the synchronization record table.

[0017] For example, the method further includes: obtaining the startup input parameter corresponding to each of the multiple script programs, the startup input parameter being used to indicate the script type; when the script type indicated by the startup input parameter is the primary script type, determining the corresponding script program as the primary script program, and when the script type indicated by the startup input parameter is the backup script type, determining the corresponding script program as the backup script program.

[0018] For example, before acquiring at least one data table to be synchronized in the first database, the method further includes: deleting historical log files and creating new log files; if an exception occurs during the process of acquiring at least one data table to be synchronized, acquiring synchronization information, and performing synchronization operations, writing the corresponding logs and error codes into the newly created log file according to a preset format and a preset log storage path; collecting logs and error codes from the newly created log file using a log collector; and issuing an alarm based on the collected logs and error codes.

[0019] According to another aspect of the present invention, an electronic device is provided, comprising: a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the data synchronization method described above.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, on which program instructions are stored, which, when executed, are used to perform the data synchronization method described above.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising computer program instructions, characterized in that the computer program instructions are used to perform the data synchronization method described above when running.

[0022] In the above technical solution, each script program obtains at least one data table to be synchronized from the first database and acquires the synchronization information corresponding to the previous synchronization operation for each data table. Using a backup script program, a synchronization operation is performed on the first data table to be synchronized where the difference between the first end time corresponding to the previous synchronization operation and the current time is greater than a first time threshold. The main script program then performs a synchronization operation on the second data table to be synchronized, thereby synchronizing the first and second data tables to the second database. Therefore, data synchronization using script programs does not rely on external components, offering advantages such as lightweight, flexibility, ease of configuration and maintenance. Furthermore, the backup script program can compensate for delayed or unsynchronized data, or take over the work of the main script program in case of failure, ensuring uninterrupted synchronization and guaranteeing the real-time performance and reliability of data synchronization.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0025] Figure 1 A schematic flowchart of a data synchronization method according to an embodiment of the present invention is shown;

[0026] Figure 2 A schematic flowchart illustrating the data synchronization process of a script program according to an embodiment of the present invention is shown;

[0027] Figure 3 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a subset of embodiments of the present invention.

[0029] In related technologies, data synchronization is usually achieved based on message queues, change data capture (CDC) tools, and extract transform load (ETL) tools. However, these technologies rely on too many components and have complex architectures, resulting in problems such as complex configuration, poor availability, and high operation and maintenance costs.

[0030] To at least partially solve the aforementioned technical problems, this invention provides a data synchronization method. This method utilizes each script program to obtain at least one data table to be synchronized in a first database, and to obtain the end time of the last synchronization operation for each data table. A backup script program is used to perform a synchronization operation on the first data table to be synchronized (the lagging data table), and a main script program is used to perform a synchronization operation on the second data table to be synchronized (the normal, non-lagging data table), thereby synchronizing the first and second data tables to be synchronized to a second database. This method implements data synchronization through script programs, without relying on external components. It has a simple and efficient architecture, is easy to maintain, and employs a master-slave mechanism. When the main script program malfunctions or causes synchronization delays for some data tables, the backup script program can automatically catch up, ensuring high availability of the entire data synchronization system. The data synchronization method can be applied to any electronic device, i.e., executed by any electronic device. Specifically, the data synchronization method can be applied to the processor of any electronic device, i.e., executed by the processor of any electronic device.

[0031] For example, Figure 1 A schematic flowchart of a data synchronization method according to an embodiment of the present invention is shown. Figure 1 As shown, the data synchronization method includes steps S1100 and S1200.

[0032] In step S1100, each of the multiple script programs is used to obtain at least one data table to be synchronized in the first database, and for each of the at least one data table to be synchronized, the synchronization information corresponding to that data table in the previous synchronization operation is obtained. The multiple script programs include a main script program and backup script programs, and the synchronization information includes the end time of the data synchronized in the corresponding synchronization operation.

[0033] A script can be a collection of independently executable program code written to achieve data synchronization. A script can achieve data synchronization without the need for other external components. Scripts can include shell scripts, Python scripts, etc. Any two scripts can be deployed on the same or different machines. Preferably, different scripts can be deployed on different machines, and the failure of one machine will not affect the operation of scripts on the remaining machines. Scripts can be managed as services. In other words, scripts can be set as services for management. Multiple scripts can include one main script and at least one backup script. The main script and backup scripts can have the same functionality and can both be used for data synchronization. When a main script malfunctions and causes a data synchronization delay, the backup script can compensate for the delayed data synchronization, ensuring smooth data synchronization.

[0034] The data synchronization method can synchronize data from a first database to a second database; that is, the first database can be the source of the data, and the second database can be the destination. Both the first and second databases can be of any type. For example, the first database can be a transactional database, such as TiDB or Spanner. Similarly, the second database can be an analytical database, such as Doris or Greenplum.

[0035] Each of the multiple script programs can obtain at least one data table to be synchronized in a first database. In some embodiments, the script program can be configured with the table name of the data table to be synchronized, and at least one data table to be synchronized can be determined based on the table name. In other embodiments, the script program can be configured with filtering conditions for the data tables to be synchronized. The filtering conditions may include table prefixes, service names, etc. The script program can filter the first database according to the filtering conditions to obtain at least one data table to be synchronized. The data table to be synchronized can be used to represent the data table to which the data to be synchronized belongs in the first database. During the synchronization operation, the data in the data table to be synchronized can be synchronized. In some embodiments, the main script program can query the first database to determine at least one data table to be synchronized and send the information of at least one data table to be synchronized to the backup script program, which does not need to query the first database. In other embodiments, each of the multiple script programs can independently query the first database to obtain at least one data table to be synchronized that it needs.

[0036] After acquiring at least one data table to be synchronized, the script can obtain the synchronization information for each table in the previous synchronization operation. This synchronization information includes the end time of the data synchronized in the corresponding synchronization operation. The synchronization information can be used to describe the execution status of the synchronization operation. It can be understood that data in a data table can have an update time, i.e., the time the data has been stored in the data table. The end time of the data synchronized in the synchronization operation can represent the latest update time of the data synchronized in that operation. In other words, a synchronization operation can synchronize data in the data table whose update time falls within a specific time range, and the end time is the latest time within that specific time range. For example, if a synchronization operation synchronizes data in the data table whose update time falls between 10:20 and 10:30, the end time is 10:30.

[0037] Each script can obtain the end time of the last synchronization operation for each data table to be synchronized. The data table to be synchronized may have undergone multiple synchronization operations previously, meaning it can have multiple synchronization records. The latest end time among these records can be determined as the synchronization record corresponding to the last synchronization operation, thus determining the end time of that last synchronization operation. Synchronization information can be recorded in either a first database or a second database. The script can query synchronization information in either the first or second database.

[0038] In step S1200, a backup script program is used to perform a synchronization operation on the first data table to be synchronized, and a main script program is used to perform a synchronization operation on the second data table to be synchronized, so as to synchronize the first and second data tables to be synchronized to the second database. The first data table to be synchronized is the data table whose difference between the first end time corresponding to the previous synchronization operation and the current time is greater than a first time threshold. The second data table to be synchronized is at least one data table other than the first data table to be synchronized.

[0039] Each script can be configured with the addresses, user information, and port numbers of the first and second databases to establish a connection and perform data synchronization. User information may include, for example, a username and password. The first end time of the data table to be synchronized can be the end time of that data table in the previous synchronization operation.

[0040] In some embodiments, for each data table to be synchronized, each script program can determine whether the difference between the first end time and the current time of the data table to be synchronized is greater than a first time threshold. Data tables to be synchronized with a difference greater than the first time threshold are designated as the first data tables to be synchronized, and data tables to be synchronized with a difference less than or equal to the first time threshold are designated as the second data tables to be synchronized. For example, the main script program can determine the difference between the first end time and the current time of each data table to be synchronized, and perform a synchronization operation on data tables to be synchronized with a difference less than or equal to the first time threshold; that is, the main script program performs a synchronization operation on the second data tables to be synchronized. Similarly, the backup script program can determine the difference between the first end time and the current time of each data table to be synchronized, and perform a synchronization operation on data tables to be synchronized with a difference greater than the first time threshold; that is, the backup script program performs a synchronization operation on the first data tables to be synchronized. The first time threshold can be set to any suitable size as needed.

[0041] In other embodiments, a data table to be synchronized where the difference between the first end time of the previous synchronization operation and the current time is greater than a first time threshold indicates that the main script program has not successfully performed the synchronization operation on that data table. The main script program can record the data tables to be synchronized that failed to perform the synchronization operation and does not need to perform the synchronization operation on those data tables again. Therefore, the main script program does not need to determine whether the difference corresponding to each data table to be synchronized is greater than the first time threshold; it only needs to perform the synchronization operation on the data tables to be synchronized that can be synchronized (i.e., the second data table to be synchronized). The backup script program can determine whether the corresponding difference for each data table to be synchronized is greater than the first time threshold and perform the synchronization operation on the data tables to be synchronized where the difference is greater than the first time threshold (i.e., the first data table to be synchronized).

[0042] The synchronization operation can synchronize data from the table to be synchronized that occurred after the first end time of the last synchronization operation to the second database. For example, it can synchronize all data in the table to be synchronized whose update time is within the range from the first end time to the current time to the second database. Alternatively, it can determine a synchronization time range based on the first end time and synchronize data in the table to be synchronized whose update time is within the synchronization time range to the second database.

[0043] In the above technical solution, each script program obtains at least one data table to be synchronized from the first database and acquires the synchronization information corresponding to the previous synchronization operation for each data table. Using a backup script program, a synchronization operation is performed on the first data table to be synchronized where the difference between the first end time corresponding to the previous synchronization operation and the current time is greater than a first time threshold. The main script program then performs a synchronization operation on the second data table to be synchronized, thereby synchronizing the first and second data tables to the second database. Therefore, data synchronization using script programs does not rely on external components, offering advantages such as lightweight, flexibility, ease of configuration and maintenance. Furthermore, the backup script program can compensate for delayed or unsynchronized data, or take over the work of the main script program in case of failure, ensuring uninterrupted synchronization and guaranteeing the real-time performance and reliability of data synchronization.

[0044] For example, the script can filter data in the first database based on table prefixes to obtain at least one table to be synchronized. It can be understood that all tables in the first database with the same table prefix can be selected as tables to be synchronized, and synchronization operations can be performed on all tables with the same table prefix. In other words, all tables in the first database with the same table prefix can be synchronized to the second database, thus achieving table aggregation synchronization. For example, for each table to be synchronized, data within different time ranges can be filtered based on update time in different synchronization operations to achieve data sharding synchronization.

[0045] For example, the script can use a preset Structured Query Language (SQL) to query the first table to be synchronized in the first database.

[0046] For example, the synchronization information also includes the start time of the data synchronized in the corresponding synchronization operation. It can be understood that data in a data table may have an update time, that is, the time the data table has stored the data. The start time of the data synchronized in the synchronization operation can represent the earliest update time of the data synchronized in that operation. In other words, a synchronization operation can synchronize data in a data table whose update time falls within a specific time range, and the start time is the earliest time within that specific time range. For example, if a synchronization operation synchronizes data in a data table whose update time falls within the range of 10:20 to 10:30, the start time is 10:20.

[0047] The synchronization operation includes steps S2100 and S2200.

[0048] In step S2100, based on the first start time or first end time of the data table to be synchronized in the previous synchronization operation, the second start time and second end time corresponding to the current synchronization operation are determined. For example, to ensure the continuity of synchronized data, the first start time or first end time can be used as the second start time based on the synchronization status of the previous synchronization operation. For example, if the previous synchronization operation failed, the first start time can be used as the second start time; if the previous synchronization operation succeeded, the first end time can be used as the second start time. In some embodiments, after determining the second start time, a first preset duration can be added to the second start time to determine the second end time. The first preset duration can be set as needed. In other embodiments, the second preset duration can be subtracted from the current time to determine the second end time. Alternatively, the current time can be used as the second end time.

[0049] In step S2200, data whose update time falls between the second start time and the second end time in the data table to be synchronized is synchronized to the second database. In some embodiments, the script program can pull data whose update time falls between the second start time and the second end time from the data table to be synchronized in the first database, and write the pulled data into the second database, thereby achieving data synchronization. In other embodiments, the script program can control the second database, which can directly query the first database for data whose update time falls between the second start time and the second end time, and directly write the queried data into the second database, thereby achieving data synchronization.

[0050] In the above technical solution, a second start time and a second end time are determined based on the first start time or the first end time corresponding to the previous synchronization operation of the data table to be synchronized. Data in the data table to be synchronized whose update time falls between the second start time and the second end time are then synchronized to the second database. Thus, incremental synchronization is achieved through the second start time and the second end time, eliminating the need to synchronize all data in the data table to be synchronized to the second database, effectively reducing the load of a single synchronization operation and lowering synchronization latency.

[0051] For example, step S2200 may include steps S2210 and S2220.

[0052] In step S2210, a connection is established between the second database and the first database using an external directory service. The second database may include an external directory service. The external directory service may be a JDBC Catalog. The second database can use the external directory service to establish a connection with the first database. For example, a script can configure the external directory service based on the address, user information, etc., of the first database to establish a connection between the second and first databases. For instance, the script can configure the data source username parameter in the external directory service to the username of the first database, the data source password parameter to the password of the first database, etc.

[0053] In step S2220, an external directory service is used to query the first database for data in the data table to be synchronized whose update time falls between the second start time and the second end time, and then writes the queried data into the second database. For example, a script can send the second start time and the second end time to the external directory service. The external directory service can then query the first database for data in the data table to be synchronized whose update time falls between the second start time and the second end time, and write the queried data into the second database to achieve data synchronization. The script can also send a structured query language (SCL) message to the external directory service to control it to query the first database for data in the data table to be synchronized whose update time falls between the second start time and the second end time, and then write the queried data into the second database.

[0054] In the above technical solution, an external directory service deployed in the second database is used to query data in the data table to be synchronized that has an update time between the second start time and the second end time in the first database, and then the queried data is written to the second database. Thus, a direct connection can be established between the second database and the first database for data synchronization. The script program does not need to fetch data and then write it to the second database; the script program is only used to control which data is synchronized, ensuring the stability and security of data synchronization.

[0055] For example, the synchronization information also includes the synchronization status of the corresponding synchronization operation. The synchronization status can be an identifier of the result of the synchronization operation. The synchronization status can be used to indicate whether the synchronization operation was successful, failed, or is being executed (i.e., synchronizing).

[0056] Step S2100 determines the second start time and second end time corresponding to the current synchronization operation based on the first start time or first end time of the data table to be synchronized in the previous synchronization operation. This may include steps S2110, S2120, and S2130. It can be understood that either step S2110 or step S2120 can be executed.

[0057] In step S2110, when the synchronization status indicates successful synchronization, the second start time is determined based on the first end time. When the synchronization status of the data table to be synchronized indicates successful synchronization in the synchronization information corresponding to the previous synchronization operation, it means that the previous synchronization operation successfully synchronized the data in the data table to be synchronized whose update time falls between the first start time and the first end time to the second database. Therefore, synchronization of data in the data table to be synchronized whose update time is after the first end time can begin. Thus, the second start time can be determined as the first end time.

[0058] In step S2120, when the synchronization status indicates synchronization failure, the second start time is determined based on the first start time. If the synchronization status of the data table to be synchronized indicates synchronization failure in the synchronization information corresponding to the previous synchronization operation, it means that the previous synchronization operation failed to synchronize data in the data table whose update time falls between the first start time and the first end time to the second database. Therefore, it is necessary to synchronize data in the data table whose update time is after the first start time. Thus, the second start time can be determined as the first start time.

[0059] In step S2130, after determining the second start time, a second end time is determined based on the second start time and the first preset duration. After determining the second start time, the sum of the second start time and the first preset duration can be used to determine the second end time. The first preset duration can be set as needed. Optionally, if the second end time is later than the current time, the second end time can be updated to the current time.

[0060] In the above technical solution, when the synchronization status indicates successful synchronization, the second start time is determined based on the first end time; when the synchronization status indicates failed synchronization, the second start time is determined based on the first start time; and the second end time is determined based on the second start time and the first preset duration. For failed synchronization operations, automatic continuation is possible, enabling breakpoint resumption and fault self-healing of data synchronization, thus improving the robustness and reliability of data synchronization.

[0061] For example, step S2130, which determines the second end time based on the second start time and the first preset duration, may include steps S2131 and S2132. It is understood that either step S2131 or step S2132 may be performed.

[0062] In step S2131, if the first reference time is later than or equal to the second reference time, the second end time is determined as the second reference time. In step S2132, if the first reference time is earlier than the second reference time, the second end time is determined as the first reference time. The first reference time is the sum of the second start time and the first preset duration, and the second reference time is the difference between the current time and the second preset duration. After determining the second start time, the sum of the second start time and the first preset duration can be calculated as the first reference time. The difference between the current time and the second preset duration can also be calculated as the second reference time. The second preset duration can be a safety buffer to prevent the synchronization progress from catching up with the latest data currently being written in real-time to the first database, thus avoiding synchronization data corruption. The first and second preset durations can be set to any suitable size as needed, and they can be equal or unequal. Specifically, the first preset duration can be any duration greater than 0, and the second preset duration can be any duration greater than 0.

[0063] If the first reference time is later than or equal to the second reference time, it means that the time range corresponding to this synchronization operation is at least partially within the safe buffer. Therefore, the second end time can be determined as the second reference time to ensure that the time range corresponding to this synchronization operation does not enter the safe buffer.

[0064] If the first reference time is earlier than the second reference time, it means that the time range corresponding to this synchronization operation has not entered the safe buffer. Therefore, the first reference time can be used as the second end time.

[0065] For example, if the second start time is 18:00, the first preset duration is 30 minutes, the second preset duration is 10 minutes, and the current time is 18:35, then the first reference time is 18:30, and the second reference time is 18:25. Since the first reference time is later than the second reference time, the second end time can be set to 18:25. As another example, assuming the second start time is 18:00, the first preset duration is 30 minutes, and the second preset duration is 10 minutes, but the current time is 18:45, then the first reference time is 18:30, and the second reference time is 18:35. Since the first reference time is earlier than the second reference time, the second end time can be set to 18:30.

[0066] In the above technical solution, the first reference time is the sum of the second start time and the first preset duration, and the second reference time is the difference between the current time and the second preset duration. If the first reference time is later than or equal to the second reference time, the second end time is determined as the second reference time; if the first reference time is earlier than the second reference time, the second end time is determined as the first reference time. Therefore, it can be ensured that the data currently being written (unstable data) is not synchronized, thus guaranteeing data security.

[0067] For example, step S1200 uses a backup script program to perform a synchronization operation on the first data table to be synchronized, and uses a main script program to perform a synchronization operation on the second data table to be synchronized, which may include step S1210.

[0068] In step S1210, each of the at least one data table to be synchronized is traversed. For the currently traversed data table, if the synchronization status of the data table indicates successful or failed synchronization, a synchronization operation is performed using a backup script or a main script. In some embodiments, the traversal can be performed according to the order in which the data tables to be synchronized are obtained. For example, the at least one data table to be synchronized may include data table 1, data table 2, and data table 3, and the acquisition order may be data table 2, data table 3, and data table 1. Therefore, the traversal can be performed in the order of 2, 3, 1. In other embodiments, the traversal order can be determined based on the dependencies between the data tables to be synchronized in the at least one data table. If there are no dependencies, the traversal order can be determined randomly.

[0069] Both the main script and the backup script can iterate over at least one data table to be synchronized. The main script executes a synchronization operation when it encounters the second data table that was previously synchronized successfully or unsuccessfully. The backup script executes a synchronization operation when it encounters the first data table that was previously synchronized successfully or unsuccessfully. In other words, for data tables that were previously synchronized, a new synchronization operation can be avoided to prevent interrupting the ongoing synchronization process.

[0070] The data synchronization method may include step S1300. In step S1300, when the synchronization status indicates "synchronization in progress," the next data table to be synchronized is traversed. It can be understood that the synchronization status corresponding to the previous synchronization operation indicates "synchronization in progress," meaning the previous synchronization operation is still in execution. When the script program traverses to a data table where the synchronization operation is still in progress, it can directly traverse the next data table to be synchronized, without performing synchronization operations on the data tables where the synchronization operation is still in progress (i.e., the synchronization status indicates "synchronization in progress").

[0071] In the above technical solution, each of at least one data table to be synchronized is traversed. For the currently traversed data table, if the synchronization status indicates success or failure, a synchronization operation is performed using a backup script or the main script. If the synchronization status indicates synchronization in progress, the traversal of the next data table to be synchronized begins. Thus, new synchronization operations are skipped for data tables that are currently being synchronized, preventing interruptions to ongoing synchronization operations and ensuring continuous and stable data synchronization.

[0072] For example, when the synchronization status of the last synchronization operation on the data table to be synchronized is "synchronizing," the difference between the end time of the most recent successful synchronization operation and the current time can be calculated. If the difference is greater than a second time threshold, a first error code is output and an alarm is triggered. This can be understood as a warning that a fault exists and maintenance is required for a data table that has been in synchronization for a long time. If the difference is less than or equal to the second time threshold, a second error code is output and an alarm is triggered. For a data table that has been in synchronization for a short time, a warning of potential synchronization delay can be triggered, requiring maintenance. The first and second error codes can be preset and can be differentiated as needed to distinguish different synchronization situations. The second time threshold can be set to any suitable size, such as 2 hours.

[0073] For example, step S1210, which iterates through at least one set of data tables to be synchronized, may include step S1211. In step S1211, each of the at least one set of data tables to be synchronized is traversed according to a preset dependency order. The at least one set of data tables to be synchronized obtained from the first database may have dependencies, and can be sorted according to these dependencies to obtain a preset dependency order for the at least one set of data tables to be synchronized. For example, a topological sort can be performed based on the dependencies between the data tables to be synchronized to obtain a preset dependency order for the at least one set of data tables to be synchronized. For instance, if the primary key `user_id` of the user table is a foreign key of the order table, then the order table depends on the user table; users must exist before orders can exist. In this case, the user table needs to be synchronized first, and then the order table. During the traversal, the traversal can be performed according to the preset dependency order. For example, at least one data table to be synchronized may include data table 1, data table 2, and data table 3, wherein data table 1 depends on data table 2, and data table 2 depends on data table 3. The preset dependency order could be data table 3, data table 2, and data table 1, thus the traversal can be performed in the order of 3, 2, 1. Optionally, the user can set the preset dependency order of at least one data table to be synchronized.

[0074] In the above technical solution, each of the at least one data table to be synchronized is traversed according to a preset dependency order. This ensures the sequential nature of data synchronization, thereby improving the robustness and success rate of data synchronization.

[0075] For example, the data synchronization method may include step S1400. In step S1400, after the step of synchronizing data in the data table to be synchronized whose update time is between the second start time and the second end time to the second database begins execution, the next data table to be synchronized begins to be traversed. That is, after step S2200 in the synchronization operation begins execution, the traversal of the next data table to be synchronized can begin without waiting for the synchronization operation of the current data table to be synchronized to complete (successfully or unsuccessfully) before traversing the next data table to be synchronized. The operation of synchronizing data in the data table to be synchronized whose update time is between the second start time and the second end time to the second database can be performed in the background. In other words, data in multiple data tables to be synchronized whose update time is between their respective second start time and second end time can be synchronized to the second database simultaneously and in parallel.

[0076] In the above technical solution, after the step of synchronizing data in the data table to be synchronized that has an update time between the second start time and the second end time to the second database begins, the next data table to be synchronized is traversed. Therefore, data synchronization can be performed on multiple data tables in parallel, achieving high throughput, improving synchronization efficiency, and reducing data synchronization latency.

[0077] For example, step S1100, obtaining the synchronization information corresponding to the previous synchronization operation of the data table to be synchronized, may include step S1110. In step S1110, a query is performed in the synchronization record table based on the table name of the data table to be synchronized. If the synchronization information corresponding to the data table to be synchronized is found, the latest synchronization information found is determined as the synchronization information corresponding to the previous synchronization operation. If the synchronization information corresponding to the data table to be synchronized is not found, an initial synchronization information entry is added to the synchronization record table. The synchronization information also includes the table name of the data table to be synchronized, and the end time in the initial synchronization information is determined based on the current time. A synchronization record table may be stored in a first database or a second database. Taking the first database as an example, the synchronization information can be queried in the synchronization record table of the first database using the table name of the data table to be synchronized. The synchronization record table may store the synchronization information corresponding to each synchronization operation for the data table to be synchronized. The synchronization information may include the table name of the data table to be synchronized, the start time and end time of the data synchronized in the corresponding synchronization operation. If the synchronization information for the data table to be synchronized is found in the synchronization record table, the latest synchronization information can be determined based on the end time of each synchronization record. In other words, among the multiple synchronization records found for the data table to be synchronized, the one with the latest end time is the latest synchronization information. The latest synchronization information found can be determined to be the synchronization information corresponding to the previous synchronization operation.

[0078] If no synchronization information for the data table to be synchronized is found in the synchronization record table, it means that no synchronization operation has been performed on the data table before. This data table to be synchronized can be a newly added data table in this round of data synchronization. If no synchronization information is found, an initial synchronization record for the data table to be synchronized can be added to the synchronization record table. The end time in the initial synchronization record can be determined based on the current time. For example, the current time can be subtracted from the third preset duration, and the result can be used as the end time in the initial synchronization record for the data table to be synchronized. The start time in the initial synchronization record can also be determined based on the current time. For example, the current time can be subtracted from the fourth preset duration, and the result can be used as the start time in the initial synchronization record for the data table to be synchronized. The third and fourth preset durations can be set to any suitable size as needed, with the fourth preset duration being longer than the third preset duration, for example, the fourth preset duration is 60 minutes and the third preset duration is 50 minutes. It can be understood that although initial synchronization information has been created, no actual synchronization operation has been performed previously; that is, the initial synchronization information does not have a corresponding actual synchronization operation. In some embodiments, the second start time for this synchronization operation can be determined without relying on the start or end time in the initial synchronization information, and the synchronization operation can be performed on the data table to be synchronized. Since the synchronization information corresponding to the data table to be synchronized can be queried in the next round of data synchronization, the synchronization operation can be performed. In other embodiments, the synchronization operation can be performed directly on the data table to be synchronized based on the initial synchronization information.

[0079] The synchronization operation also includes step S2300. In step S2300, a new synchronization information record corresponding to this synchronization operation is added to the synchronization record table. Each time the script program executes a synchronization operation, a new synchronization information record corresponding to this synchronization operation can be added to the synchronization record table. The synchronization information can be added to the synchronization record table based on the table name of the data table to be synchronized, the second start time, and the second end time corresponding to this synchronization operation. When adding the information, the synchronization status in the synchronization information can be set to "synchronizing in progress." When the synchronization operation ends, the "synchronizing in progress" status can be changed to "synchronization successful" or "synchronization failed." The synchronization information can also record the execution start time, execution end time, total execution time, synchronization information update or addition time, first database name, second database name, etc., corresponding to this synchronization operation.

[0080] In the above technical solution, if the synchronization information corresponding to the data table to be synchronized is found in the synchronization record table, the latest synchronization information found is determined as the synchronization information corresponding to the previous synchronization operation. If no synchronization information is found, an initial synchronization information record is added to the synchronization record table. When a synchronization operation is executed, a new synchronization information record corresponding to this synchronization operation is added to the synchronization record table. Thus, recording each synchronization operation facilitates operation and maintenance, and allows for adaptive handling of newly added data tables to be synchronized, improving the robustness and usability of the system.

[0081] For example, the data synchronization method further includes steps S1500 and S1600. In step S1500, the startup parameters corresponding to each of the multiple script programs are obtained. The startup parameters are used to indicate the script type. In some embodiments, when deploying the script programs, corresponding startup parameters can be configured for each script program. The startup parameters may include leader or slave. A startup parameter of leader can indicate that the script type is a primary script type, and a startup parameter of slave can indicate that the script type is a backup script type.

[0082] In step S1600, when the script type indicated by the startup input parameter is the primary script type, the corresponding script program is determined to be the primary script program; when the script type indicated by the startup input parameter is the secondary script type, the corresponding script program is determined to be the secondary script program. The code of the primary script program and the secondary script program can be the same, but the startup input parameters are different. For example, the startup input parameter of the primary script program can be "leader," and the startup input parameter of the secondary script program can be "slave." In other words, a script program can be used as the primary script program when the script type indicated by the startup input parameter is the primary script type. Similarly, a script program can be used as the secondary script program when the script type indicated by the startup input parameter is the secondary script type.

[0083] In the above technical solution, when the script type indicated by the startup input parameter is the primary script type, the corresponding script program is determined to be the primary script program; when the script type indicated by the startup input parameter is the backup script type, the corresponding script program is determined to be the backup script program. Therefore, the primary and backup roles of the script programs can be assigned simply through the startup input parameters, without modifying the internal code of the script programs, greatly reducing the complexity of operation and maintenance.

[0084] For example, before obtaining at least one data table to be synchronized in the first database in step S1100, the data synchronization method further includes step S1700. Step S1700 may be performed before step S1100.

[0085] In step S1700, historical log files are deleted and new log files are created. Historical log files can be log files generated from the last or earlier data synchronization method execution. These historical log files record past execution states and consume significant storage space over time. Therefore, deleting historical log files saves storage space. After deleting historical log files, new log files can be created to record the execution state of the current data synchronization method. Log directories and log files can be automatically created according to a preset log storage path.

[0086] The data synchronization method also includes steps S1800, S1900 and S2000.

[0087] In step S1800, if an exception occurs during the process of acquiring at least one data table to be synchronized, acquiring synchronization information, and performing synchronization operations, the corresponding log and error code are written to a newly created log file according to a preset format and preset log storage path. The preset format can be a standardized format required for generating logs. For example, a log entry may include a timestamp, log level, class name, etc. For instance, the preset format could be timestamp + [service name] + [task type] + [log level] + [class name] + [task identifier] + specific log content. The preset format can be set as needed. The error code can be a unique identifier representing a specific exception type. Different types of exceptions have different error codes. Short and standardized error codes facilitate quick matching and classification by the alarm system. The preset log storage path can be used to determine the location of the log file. For example, if acquiring at least one data table to be synchronized fails, the corresponding log and error code can be written to the log file. If acquiring synchronization information corresponding to each data table to be synchronized fails, the corresponding log and error code can be written to the log file. If the synchronization operation fails, the corresponding log and error code can be written to the log file. It's understandable that any exception occurring during the execution of an operation in a script program will result in a corresponding log entry and error code being written to the log file. Different operations will have different log entries and error codes.

[0088] In step S1900, a log collector is used to collect logs and error codes from the newly created log file. The log collector can be used to monitor the log file in real time; once new log content is detected, it can be read and transmitted to the log analysis platform. For example, the log collector can be Filebeat, Fluentd, etc. For example, the log analysis platform can be ELK.

[0089] In step S2000, an alarm is generated based on the collected logs and error codes. The log analysis platform can be connected to the log alarm system. Therefore, alarms can be generated based on the collected logs and error codes. The collected logs and error codes can be analyzed, and alarm strategies can be matched according to preset rules. Based on the matched alarm strategies, corresponding alarm information can be issued. The alarm information may include the error code, the occurrence time, etc.

[0090] In the above technical solution, historical log files are deleted and new log files are created. If an anomaly is found, the corresponding log and error code are written to the newly created log file according to a preset format and preset log storage path. A log collector is used to collect logs and error codes from the newly created log file. Alarms are generated based on the collected logs and error codes. This not only avoids the problem of historical log files occupying storage space but also ensures that faults can be detected and handled promptly.

[0091] For example, Figure 2A schematic flowchart illustrating the data synchronization process of a script program according to an embodiment of the present invention is shown. The script program can obtain startup input parameters and perform parameter validation. Parameter validation can verify startup input parameters, the addresses of the first and second databases, user information, port numbers, etc. If parameter validation fails, it terminates abnormally, outputs the corresponding log and error code, and issues an alarm. If parameter validation succeeds, it can delete historical log files, create new log files, and output subsequent logs to the newly created log files. It can read configuration files, obtain SQL statements, and execute SQL statements to obtain at least one data table to be synchronized. If obtaining the data table to be synchronized fails, it terminates abnormally, outputs the corresponding log and error code, and issues an alarm. If obtaining at least one data table to be synchronized is successful, it can refresh the external directory and establish a connection between the first and second databases. If establishing the connection fails, it terminates abnormally, outputs the corresponding log and error code, and issues an alarm. If establishing the connection is successful, it iterates through each of the at least one data table to be synchronized and obtains the synchronization information corresponding to that data table from the synchronization record table. If the query fails, it outputs the corresponding log and error code and issues an alarm. If the query is successful but no synchronization information is found for the data table to be synchronized, an initial synchronization record is added to the synchronization record table. If the addition is successful, the next data table to be synchronized is traversed. If the addition fails, the corresponding log and error code are output, and an alarm is triggered. If synchronization information is found for the data table to be synchronized, the script program processes it according to the startup parameters. If the script type indicated by the startup parameters is the main script type, the synchronization status of the data table to be synchronized is obtained. If the retrieval fails, the corresponding log and error code are output, and an alarm is triggered. If the retrieval is successful, the second start time is determined based on the first end time when the synchronization status indicates successful synchronization. The second start time is determined based on the first start time when the synchronization status indicates failed synchronization. The corresponding log and error code are output, and an alarm is triggered when the synchronization status indicates synchronization is in progress. After determining the second start time, the second end time is determined based on the second start time and the first preset duration. Furthermore, a new synchronization record corresponding to this synchronization operation is added to the synchronization log table. If the addition fails, the corresponding log and error code are output, and an alarm is triggered. If the addition is successful, the external directory service is used to query the data in the table to be synchronized in the first database that has an update time between the second start time and the second end time, and the queried data is written to the second database. If this fails, the corresponding log and error code are output, and an alarm is triggered. If successful, the next data table to be synchronized can be traversed. After outputting the corresponding log and error code and triggering the alarm, the next data table to be synchronized can be directly traversed.If the script type indicated by the startup input parameter of the script program is the backup script type, then it is determined whether the difference between the first end time and the current time of the data table to be synchronized is greater than the first time threshold. If yes, the data table to be synchronized can be synchronized in the same way as the main script program. If no, the data synchronization can be skipped and the next data table to be synchronized can be traversed directly.

[0092] By way of example, according to another aspect of the present invention, a data synchronization apparatus is also provided. The data synchronization apparatus includes a processor. The processor is used to execute the data synchronization method of any embodiment of the present application.

[0093] By way of example, according to another aspect of the present invention, an electronic device is also provided. Figure 3 A schematic block diagram of an electronic device 300 according to an embodiment of the present invention is shown. The electronic device 300 includes a processor 310 and a memory 320. The memory 320 stores computer program instructions, which are executed by the processor 310 to perform the data synchronization method described above.

[0094] By way of example, according to another aspect of the present invention, a computer-readable storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the data synchronization method described above. The computer-readable storage medium may, for example, include an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above-described computer-readable storage media.

[0095] By way of example, according to another aspect of the present invention, a computer program product is also provided, including computer program instructions that, when executed, are used to perform the data synchronization method as described above.

[0096] Those skilled in the art can understand the specific implementation schemes and beneficial effects of the above-mentioned data synchronization devices, electronic devices, computer-readable storage media, and computer program products by reading the relevant descriptions of the data synchronization methods above, and for the sake of brevity, they will not be described in detail here.

[0097] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0100] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0101] Similarly, it should be understood that, in order to streamline this application and aid in understanding one or more of the various inventive aspects, features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with features fewer than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0102] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0103] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0104] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the data synchronization apparatus according to the embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0105] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0106] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A data synchronization method, characterized in that, include: Using each of the multiple script programs, at least one data table to be synchronized in the first database is obtained, and for each of the at least one data table to be synchronized, the synchronization information corresponding to the previous synchronization operation is obtained. The multiple script programs include a main script program and a backup script program, and the synchronization information includes the end time of the data synchronized in the corresponding synchronization operation. Using the backup script program, a synchronization operation is performed on the first data table to be synchronized, and using the main script program, the synchronization operation is performed on the second data table to be synchronized, so as to synchronize the first data table to be synchronized and the second data table to be synchronized to the second database. Wherein, the first data table to be synchronized is the data table to be synchronized where the difference between the first end time corresponding to the last synchronization operation and the current time is greater than the first time threshold, and the second data table to be synchronized is the data table to be synchronized other than the first data table to be synchronized in the at least one data table to be synchronized.

2. The data synchronization method according to claim 1, characterized in that, The synchronization information also includes the start time of the data synchronized in the corresponding synchronization operation; The synchronization operation includes: Based on the first start time or first end time of the previous synchronization operation for the data table to be synchronized, determine the second start time and second end time for this synchronization operation. Data in the data table to be synchronized whose update time is between the second start time and the second end time will be synchronized to the second database.

3. The data synchronization method according to claim 2, characterized in that, The step of synchronizing data in the data table to be synchronized whose update time falls between the second start time and the second end time to the second database includes: Establish a connection between the second database and the first database using an external directory service; The external directory service is used to query the data in the data table to be synchronized in the first database whose update time is between the second start time and the second end time, and the queried data is written into the second database.

4. The data synchronization method according to claim 2, characterized in that, The synchronization information also includes the synchronization status of the corresponding synchronization operation; The step of determining the second start time and second end time corresponding to the current synchronization operation based on the first start time or first end time of the data table to be synchronized in the previous synchronization operation includes: When the synchronization status indicates successful synchronization, the second start time is determined based on the first end time. When the synchronization state indicates synchronization failure, the second start time is determined based on the first start time; After determining the second start time, the second end time is determined based on the second start time and the first preset duration.

5. The data synchronization method according to claim 4, characterized in that, The step of using the backup script program to perform a synchronization operation on the first data table to be synchronized, and using the main script program to perform the synchronization operation on the second data table to be synchronized, includes: The system iterates through each of the at least one data tables to be synchronized. For the currently traversed data table to be synchronized, when the synchronization status of the data table indicates successful or failed synchronization, the system uses the backup script program or the main script program to perform the synchronization operation. The method further includes: When the synchronization state indicates that synchronization is in progress, the next data table to be synchronized is traversed.

6. The data synchronization method according to claim 5, characterized in that, The method further includes: after the step of synchronizing the data in the data table to be synchronized whose update time is between the second start time and the second end time to the second database is started, the next data table to be synchronized is traversed.

7. The data synchronization method according to claim 5, characterized in that, The step of traversing each of the at least one data tables to be synchronized includes: According to the preset dependency order, traverse each of the at least one data tables to be synchronized.

8. The data synchronization method according to claim 4, characterized in that, The step of determining the second end time based on the second start time and the first preset duration includes: If the first reference time is later than or equal to the second reference time, then the second end time shall be determined as the second reference time; If the first reference time is earlier than the second reference time, then the second end time is determined as the first reference time; Wherein, the first reference time is the sum of the second start time and the first preset duration, and the second reference time is the difference between the current time and the second preset duration.

9. An electronic device, comprising: Processor and memory, characterized in that, The memory stores computer program instructions, which, when executed by the processor, are used to perform the data synchronization method as described in any one of claims 1 to 8.

10. A computer-readable storage medium on which program instructions are stored, characterized in that, The program instructions are used to execute the data synchronization method as described in any one of claims 1 to 8 when the program is run.

11. A computer program product comprising computer program instructions, characterized in that, The computer program instructions are used to execute the data synchronization method as described in any one of claims 1 to 8 when the program is run.