Data table synchronization method, apparatus, device, and storage medium
By selecting the incremental task with the shortest export time and the full task with the longest export time during the data table synchronization process, and determining whether they are the same data table, the full task is executed first, thus solving the problem of low data table synchronization efficiency in the existing technology and achieving more efficient and accurate data table updates.
Patent Information
- Application Number
- CN202210989399.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-17
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2042-08-17
AI Technical Summary
In the prior art, when electronic devices receive multiple incremental synchronization tasks and full synchronization tasks, they execute them sequentially based on the order in which the tasks were generated, resulting in low data table synchronization efficiency.
Among the pending tasks corresponding to the data table, select the incremental task with the shortest export time and the full task with the longest export time, and determine whether they are from the same data table. If they are the same, execute the full task first and ignore the earlier incremental tasks. Improve synchronization efficiency through a mixed sorting method.
This improves the synchronization efficiency of data tables, avoids the problem of low-frequency data tables failing to update in a timely manner when update frequencies are inconsistent, and ensures the accuracy and efficiency of data table updates.
Smart Images

Figure CN115292416B_ABST
Abstract
Description
Technical Field
[0001] This application relates to big data technology, and in particular to a data table synchronization method, apparatus, device, and storage medium. Background Technology
[0002] Currently, with the continuous development of information technology, enterprise personnel often need to synchronize a large number of data tables to provide a data foundation for subsequent business analysis.
[0003] In related technologies, when an electronic device receives multiple incremental synchronization tasks and full synchronization tasks, it executes each received task sequentially based on the order of their generation times, resulting in low data synchronization efficiency.
[0004] Therefore, a data table synchronization method is needed to improve the synchronization efficiency of data tables. Summary of the Invention
[0005] This application provides a data table synchronization method, apparatus, device, and storage medium to solve the problem of low data table synchronization efficiency in related technologies.
[0006] Firstly, this application provides a data table synchronization method, including:
[0007] Among the incremental tasks to be processed corresponding to each data table, the incremental task with the shortest first export time is determined as the first task; wherein, the incremental task is used to indicate that incremental update processing is performed on the data table corresponding to the incremental task; the first export time is the generation time of the incremental task;
[0008] Among the pending full tasks corresponding to each data table, the full task with the longest second export time is determined as the second task; wherein, the full task is used to instruct a full update process for the data table indicated by the full task; the second export time is the generation time of the full task;
[0009] If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the second task is executed.
[0010] In some embodiments, if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the second task is executed, including:
[0011] If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the second task are determined to be the second task group.
[0012] If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, then the second task is executed.
[0013] In some embodiments, if it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, then the second task is executed, including:
[0014] If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the second task, then the second task is executed.
[0015] In some embodiments, the method further includes:
[0016] If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the first task is executed.
[0017] In some embodiments, the incremental task has a task sequence number, which indicates the generation order of the incremental tasks among all incremental tasks of the data table indicated by the incremental task; if it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the first task is executed, including:
[0018] If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0019] Based on the task number of the first task, determine the previous task number of the task number of the first task, wherein the previous task number is the task number of the incremental task corresponding to the previous generation order of the generation order of the first task in the generated incremental tasks corresponding to the data table.
[0020] If it is determined that the completed incremental tasks include an incremental task with the previous task number, then the first task is executed.
[0021] In some embodiments, the method further includes:
[0022] If it is determined that the completed incremental tasks do not include an incremental task with the previous task number, then the step of determining the incremental task with the shortest first export time as the first task among the pending incremental tasks corresponding to each data table is repeated.
[0023] In some embodiments, if it is determined that the completed incremental tasks do not include an incremental task with the previous task number, then the step of determining the incremental task with the shortest first export time as the first task among the pending incremental tasks corresponding to each data table is repeated, including:
[0024] If it is determined that the completed incremental tasks do not include incremental tasks with the previous task number, then the current time is subtracted from the receiving time of the first task to obtain the time difference.
[0025] If the time difference is less than or equal to a preset threshold, then the step of determining the incremental task with the smallest first export time as the first task among the incremental tasks to be processed in each data table is repeated.
[0026] In some embodiments, the method further includes:
[0027] If the time difference is greater than the preset threshold, then the first task is executed.
[0028] In some embodiments, if it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then determining the completed incremental task corresponding to the data table indicated by the first task includes:
[0029] If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the first task are determined to be the first task group.
[0030] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0031] In some embodiments, the method further includes:
[0032] If it is determined that there is a full set of tasks in the first task group whose second export time is greater than the first export time of the first task, then the status of the first task is set to the processed status.
[0033] In some embodiments, if it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental tasks corresponding to the data table indicated by the first task are determined, including:
[0034] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0035] In some embodiments, the first task has data to be updated; the data table has a primary key field; executing the first task includes:
[0036] In the data to be updated in the first task, determine the update data corresponding to the primary key field of the data table indicated by the first task;
[0037] If it is determined that the data table includes the updated data, then the data in the row containing the updated data in the data table is replaced with the data to be updated;
[0038] If it is determined that the data table does not contain the updated data, then the data to be updated is added to the data table.
[0039] In some embodiments, the second task includes data to be updated; performing the second task includes:
[0040] Delete the data in the data table indicated by the second task, and write the data to be updated in the second task into the data table.
[0041] Secondly, this application provides a data table synchronization device, comprising:
[0042] The first determining unit is used to determine the incremental task with the shortest first export time as the first task among the incremental tasks to be processed corresponding to each data table; wherein, the incremental task is used to indicate that incremental update processing is performed on the data table corresponding to the incremental task; the first export time is the generation time of the incremental task;
[0043] The second determining unit is used to determine the full task with the longest second export time as the second task among the full tasks to be processed corresponding to each data table; wherein, the full task is used to instruct a full update process for the data table indicated by the full task; and the second export time is the generation time of the full task.
[0044] The first execution unit is configured to execute the second task if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task.
[0045] In some embodiments, the first execution unit includes:
[0046] The first determining module is used to determine the completed full tasks corresponding to the data table indicated by the second task as the second task group if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task.
[0047] The first execution module is configured to execute the second task if it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task.
[0048] In some embodiments, the first execution module is specifically used for:
[0049] If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the second task, then the second task is executed.
[0050] In some embodiments, the apparatus further includes:
[0051] The second execution unit is configured to execute the first task if it is determined that the data table indicated by the first task is different from the data table indicated by the second task.
[0052] In some embodiments, the incremental task has a task sequence number, which indicates the generation order of the incremental task among all incremental tasks in the data table indicated by the incremental task; the second execution unit includes:
[0053] The second determining module is used to determine the completed incremental task corresponding to the data table indicated by the first task if it is determined that the data table indicated by the first task is different from the data table indicated by the second task.
[0054] The third determining module is used to determine the previous task number of the first task based on the task number of the first task, wherein the previous task number is the task number of the incremental task corresponding to the previous generation order of the generation order of the first task in the generated incremental tasks corresponding to the data table.
[0055] The second execution module is configured to execute the first task if it is determined that the completed incremental tasks include an incremental task with the previous task number.
[0056] In some embodiments, the apparatus further includes:
[0057] The third execution module is used to repeatedly execute the step of determining the incremental task with the shortest first export time as the first task among the incremental tasks to be processed in each data table if it is determined that the completed incremental tasks do not include an incremental task with the previous task number.
[0058] In some embodiments, the third execution module is specifically used for:
[0059] If it is determined that the completed incremental tasks do not include incremental tasks with the previous task number, then the current time is subtracted from the receiving time of the first task to obtain the time difference.
[0060] If the time difference is less than or equal to a preset threshold, then the step of determining the incremental task with the smallest first export time as the first task among the incremental tasks to be processed in each data table is repeated.
[0061] In some embodiments, the third execution module is further configured to
[0062] If the time difference is greater than the preset threshold, then the first task is executed.
[0063] In some embodiments, the second determining module is specifically used for:
[0064] If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the first task are determined to be the first task group.
[0065] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0066] In some embodiments, the second determining module is further configured to: if it is determined that there are full tasks in the first task group whose second export time is greater than the first export time of the first task, then set the status of the first task to the processed status.
[0067] In some embodiments, when the second determining module performs the operation of determining the completed incremental task corresponding to the data table indicated by the first task if it is determined that there is no full task in the first task group whose second export time is greater than the first export time of the first task, the module is specifically used to:
[0068] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0069] In some embodiments, the first task has data to be updated; the data table has a primary key field; the second execution unit executes the first task, specifically for:
[0070] In the data to be updated in the first task, determine the update data corresponding to the primary key field of the data table indicated by the first task; if it is determined that the data table includes the update data, then replace the data in the row where the update data is located with the data to be updated; if it is determined that the data table does not include the update data, then add the data to be updated to the data table.
[0071] In some embodiments, the second task includes data to be updated; when the first execution unit executes the second task, it is specifically used for:
[0072] Delete the data in the data table indicated by the second task, and write the data to be updated in the second task into the data table.
[0073] Thirdly, this application provides an electronic device, including: a memory and a processor;
[0074] Memory; memory for storing executable instructions of the processor;
[0075] The processor is configured to execute the method as described in any of the first aspects according to the executable instructions.
[0076] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.
[0077] Fifthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the method described in any one of the first aspects.
[0078] The data table synchronization method, apparatus, device, and storage medium provided in this application select the incremental task with the shortest first export time from the incremental tasks to be processed corresponding to each data table in the electronic device, and select the full task with the longest second export time from the full tasks to be processed corresponding to each data table. It then determines whether the selected full task and the incremental task correspond to the same data table. If they correspond to the same data table, the second task can be executed directly. It is understood that when the two correspond to the same data table, the second task can be executed first, and the incremental tasks corresponding to the data table whose first export time is earlier than the second export time of the second task can be ignored, thereby improving the data table synchronization efficiency. Furthermore, in this embodiment, the incremental tasks to be processed corresponding to multiple data tables are mixed and sorted according to the first export time of each incremental task. The updates of multiple data tables can be based on the task order generated by the electronic device on the master database side. Compared to sorting the incremental tasks corresponding to each data table separately, the mixed sorting method can avoid the problem that data tables with lower update frequencies cannot be updated in a timely manner when the update frequencies of multiple data tables in the electronic device are inconsistent. Attached Figure Description
[0079] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0080] Figure 1 This application provides an illustration of an application scenario.
[0081] Figure 2 A flowchart illustrating a data table synchronization method provided in an embodiment of this application;
[0082] Figure 3 A flowchart illustrating the second data table synchronization method provided in this application embodiment;
[0083] Figure 4 A flowchart illustrating the third data table synchronization method provided in this application embodiment;
[0084] Figure 5 A schematic diagram of a data table synchronization device provided in an embodiment of this application;
[0085] Figure 6 This is a schematic diagram of the structure of a second type of data table synchronization device provided in an embodiment of this application;
[0086] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0087] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0088] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0089] Currently, when electronic devices synchronize data, the primary database device can send incremental or full tasks to the backup database device to achieve data table synchronization between the primary and backup databases. Figure 1 This is a schematic diagram illustrating an application scenario provided by this application. As shown in the figure... Figure 1 In this system, the primary database's electronic devices can periodically check for changes in the data stored on the primary database, generate files for the changed data, and send these generated files as incremental tasks to the backup database's electronic files. Furthermore, the primary database's electronic devices can also periodically generate files from the full data stored on the electronic devices and send them to the backup database's electronic devices, allowing the backup database's electronic devices to perform a full data synchronization operation upon receiving the full task. For example, if the backup database currently has full task 1, full task 2, incremental task 1, incremental task 2, and incremental task 3 yet to be executed, the backup database will execute these five tasks sequentially based on their generation time, as shown in the diagram: full task 1, incremental task 1, incremental task 2, incremental task 3, and full task 2.
[0090] However, when the primary database sends tasks to the backup database, if the backup database's electronic devices receive multiple incremental synchronization tasks and full synchronization tasks, the backup database's electronic devices will execute each received task in the order of their generation time, resulting in low data synchronization efficiency.
[0091] For example, starting at 10:00 AM, the electronic devices on the primary database generate a full file every 10 minutes, resulting in 60 full files generated by 8:00 PM. During this period, the electronic devices on the standby database are down due to an upgrade. Upon restarting at 8:00 PM, they need to process these 60 full files sequentially, leading to a large number of read and write operations in a short time. However, for full files, processing all 60 full files sequentially yields the same result as processing only the latest full file. If the electronic devices on the standby database still use the sequential processing method, the data synchronization efficiency will be low.
[0092] The data table synchronization method, apparatus, device, and storage medium provided in this application are intended to solve the above-mentioned technical problems of the prior art.
[0093] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0094] Figure 2 This is a flowchart illustrating a data table synchronization method provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps:
[0095] S201. Among the incremental tasks to be processed corresponding to each data table, the incremental task with the shortest first export time is determined as the first task; wherein, the incremental task is used to indicate the incremental update processing of the data table corresponding to the incremental task; the first export time is the generation time of the incremental task.
[0096] For example, the execution subject of this embodiment can be a data table synchronization device, which can be an electronic device or the like. This embodiment uses an electronic device as an example for description.
[0097] In this embodiment, the incremental task received by the electronic device has a first export time, which is the same as the generation time of the incremental task. Furthermore, the incremental task is used to instruct incremental update processing on the data table indicated by the incremental task.
[0098] During the data table synchronization process, when an electronic device receives incremental tasks or full tasks from other electronic devices, it selects the incremental task with the shortest first export time from among the unprocessed incremental tasks (i.e., pending incremental tasks) and designates it as the first task. The shortest first export time indicates that this incremental task was generated earliest among the pending incremental tasks.
[0099] It should be noted that the incremental tasks to be processed in this embodiment include unprocessed incremental tasks corresponding to each data table in the electronic device.
[0100] S202. Among the full tasks to be processed corresponding to each data table, the full task with the largest second export time is determined as the second task; wherein, the full task is used to indicate the full update processing of the data table indicated by the full task; the second export time is the generation time of the full task.
[0101] For example, in this embodiment, the full task received by the electronic device has a second export time, which is the generation time of the incremental task. Furthermore, the full task is used to instruct a full update of the data table indicated by the full task. The second export time is the longest, indicating that the full task was generated latest among the pending full tasks.
[0102] During the data table synchronization process, when an electronic device receives incremental or full tasks sent by other electronic devices, it will also select the full task with the longest export time from the unprocessed full tasks in the electronic device as the second task.
[0103] It should be noted that the total tasks to be processed in this embodiment include the unprocessed total tasks corresponding to each data table in the electronic device. Furthermore, this embodiment does not impose specific restrictions on the execution order of steps S201 and S202; step S202 can be executed first, followed by step S201.
[0104] S203. If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then execute the second task.
[0105] For example, after the electronic device determines the first task and the second task, it will determine whether the data table indicated by the first task and the data table indicated by the second task are the same.
[0106] In one example, the incremental task received by the electronic device has a data table identifier for the data table indicated by the incremental task. Furthermore, the full task also has a data table identifier for the data table indicated by the full task. Therefore, by comparing the data table identifiers in the first task and the second task, it can be determined whether the data table indicated by the first task and the second task is the same data table.
[0107] When the electronic device determines that the data table indicated by the first task and the second task are the same, the electronic device can then execute the second task, that is, perform a full update process.
[0108] In this embodiment, among the incremental tasks to be processed corresponding to each data table in the electronic device, the incremental task with the shortest first export time is selected, and among the full tasks to be processed corresponding to each data table, the full task with the longest second export time is selected. It is then determined whether the selected full task and the incremental task correspond to the same data table. If they are the same data table, the second task can be executed directly. It is understood that when they correspond to the same data table, the second task can be executed first, while the incremental tasks corresponding to the data table whose first export time is earlier than the second export time of the second task can be ignored, thereby improving the synchronization efficiency of the data tables. Furthermore, in this embodiment, the incremental tasks to be processed corresponding to multiple data tables are sorted in a mixed manner according to the first export time of each incremental task. Updates of multiple data tables can be performed based on the task order generated by the electronic device on the master database. Compared to sorting the incremental tasks corresponding to each data table separately, the mixed sorting method avoids the problem of data tables with lower update frequencies not being updated in a timely manner when the update frequencies of multiple data tables in the electronic device are inconsistent.
[0109] Figure 3 A flowchart illustrating the second data table synchronization method provided in this application embodiment is shown below. Figure 3 As shown, the method includes the following steps:
[0110] S301. Among the incremental tasks to be processed corresponding to each data table, the incremental task with the smallest first export time is determined as the first task; wherein, the incremental task is used to indicate the incremental update processing of the data table corresponding to the incremental task; the first export time is the generation time of the incremental task.
[0111] S302. Among the pending full tasks corresponding to each data table, determine the full task with the longest second export time as the second task; wherein, the full task is used to indicate the full update processing of the data table indicated by the full task; the second export time is the generation time of the full task.
[0112] For example, the specific principles of steps S301 and S302 can be found in steps S201 and S202, and will not be repeated here.
[0113] S303. If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the second task are determined to be the second task group.
[0114] For example, the execution subject in this embodiment is an electronic device.
[0115] In this embodiment, when the data table is synchronized, it records all the completed tasks corresponding to the data table. That is, the electronic device records the historical update records of the data table, which can store all the tasks that have been processed for the data table.
[0116] When the electronic device determines that the data table indicated by the first task and the second task is the same data table, it can search for the full tasks corresponding to the data table in the full tasks that have been processed, and take the completed full tasks corresponding to the data table as the second task group. The second task group includes all the full tasks that are currently completed corresponding to the data table.
[0117] S304. If it is determined that there is no full task in the second task group whose second export time is greater than the second export time of the second task, then execute the second task.
[0118] For example, after determining the second task group, the second export time of all completed full tasks included in the second task group is compared with the second export time of the second task. If the second export time of all full tasks included in the second task group is less than the second export time of the second task, then the second task can be executed directly.
[0119] If, among the full tasks included in the second task group, there is at least one completed full task whose second export time is greater than or equal to the second export time of the second task, it indicates that the full task whose second export time is later than the second export time of the second task has already been executed by the electronic device. In this case, there is no need to execute the second task again.
[0120] Understandably, before executing the second task, the second export time of the completed full task corresponding to the data table can be compared with the second export time of the second task. If the second export time of the second task is greater than the second export time of each full task in the second task group, the second task can be executed directly. If at least one full task in the second task group has a second export time greater than or equal to the second export time of the second task, it indicates that all data contained in the data table has been updated by a full task with a second export time later than that of the second task. Therefore, it is not necessary to execute the second task on the data table at this time. The data table update method provided in this embodiment improves the accuracy and efficiency of data table updates.
[0121] In one example, step S304 can also be achieved using the following method:
[0122] If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the second task, then the second task is executed.
[0123] For example, in this embodiment, the electronic device can be configured with multiple processing processes. Each processing process can only perform full or incremental update processing on one data table, and the data tables processed by each processing process at the same time are different. Therefore, when it is determined that there is no full task in the second task group whose second export time is greater than the second export time of the second task, it is also necessary to determine whether there is currently a process processing the data table. That is, it is necessary to determine whether the full task or incremental task corresponding to the data table is included in the currently processing full tasks and the currently processing incremental tasks. If neither the currently processing full tasks nor the currently processing incremental tasks include the full task or incremental task corresponding to the data table indicated by the second task, it indicates that there is currently no process performing full or incremental update processing on the data table indicated by the second task. Therefore, the second task can be executed directly.
[0124] In one possible implementation, if no process is currently performing a full update or incremental update on the data table indicated by the second task, step S301 can be repeated, or after waiting for a period of time, it can be determined whether the full task currently in the processing state and the incremental task currently in the processing state contain the full task and incremental task corresponding to the data table indicated by the second task, until the second task is executed.
[0125] It is understood that, in order to avoid the problem of inaccurate data table updates caused by multiple processes updating the same data table at the same time, this embodiment will also determine whether the full task and incremental task currently in the processing state contain the full task and incremental task corresponding to the data table indicated by the second task, thereby improving the accuracy of data updates during the data table synchronization update process.
[0126] In one example, the second task includes data to be updated; the step "execute the second task" in step S304 can be performed by deleting the data in the data table indicated by the second task and writing the data to be updated in the second task into the data table.
[0127] For example, in this embodiment, the second task includes data to be updated. When the electronic device executes the second task, it deletes all data in the data table indicated by the second task, and then writes the data to be updated contained in the second task into the data table indicated by the second task, thereby completing the full task update process indicated by the second task.
[0128] S305. If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then execute the first task.
[0129] For example, after the electronic device determines the first task and the second task, if it is determined that the data table indicated by the first task and the second task are different, then the first task is executed first, that is, the data table indicated by the first task is updated first.
[0130] It is understandable that when the data tables indicated by the first task and the second task are different, since the update frequency of incremental tasks for data tables is usually greater than the update frequency of full tasks for data tables, this is done in order to make the updates of the data tables in the electronic device more closely resemble the changes in the corresponding data tables on the master database.
[0131] In one example, the first task has data to be updated; the data table has a primary key field; when performing "Execute the first task" in step S305, it can be achieved through the following steps:
[0132] The first step is to determine the update data corresponding to the primary key field of the data table indicated by the first task in the data to be updated in the first task.
[0133] The second step is to replace the data in the row containing the updated data with the data to be updated if the data table is determined to contain updated data.
[0134] Third step: If it is determined that the data table does not contain the data to be updated, then add the data to be updated to the data table.
[0135] For example, in this embodiment, the first task received by the electronic device includes data to be updated. Furthermore, the data table in the electronic device also has a primary key field, where the data corresponding to the primary key field uniquely identifies a row of data in the data table. When the electronic device needs to execute the first task, it first determines the update data corresponding to the primary key field of the data table indicated by the first task from the data to be updated in the first task. Then, it searches for the update data in the data table indicated by the first task. If the data table contains the update data, the data in the row containing the update data is replaced, that is, the data to be updated in the first task is replaced in that row. It can be understood that when all fields in the data to be updated are empty except for the update data, it indicates that the row of data needs to be deleted; if it is not empty, the row of data is updated and replaced. When the data table does not contain the update data, it indicates that the data to be updated included in the first task needs to be added to the data table.
[0136] In this embodiment, before executing the second task, the second export time of the completed full task corresponding to the data table can be compared with the second export time of the second task. If at least one full task in the second task group has a second export time greater than or equal to the second export time of the second task, it indicates that all data contained in the data table has been updated by a full task with a second export time later than that of the second task. Therefore, it is not necessary to execute the second task on the data table at this time. The data table update method provided in this embodiment improves the accuracy and efficiency of data table updates. In addition, to avoid the problem of inaccurate data table updates caused by multiple processes updating the same data table simultaneously, this embodiment also determines whether the full task and incremental task currently in the processing state contain the full task and incremental task corresponding to the data table indicated by the second task, thereby improving the accuracy of data updates during the data table synchronization update process.
[0137] Figure 4 This is a flowchart illustrating the third data table synchronization method provided in this application embodiment. Figure 4 As shown, the method includes the following steps:
[0138] S401. Among the incremental tasks to be processed corresponding to each data table, the incremental task with the shortest first export time is determined as the first task; wherein, the incremental task is used to indicate the incremental update processing of the data table corresponding to the incremental task; the first export time is the generation time of the incremental task.
[0139] S402. Among the pending full tasks corresponding to each data table, determine the full task with the longest second export time as the second task; wherein, the full task is used to indicate the full update processing of the data table indicated by the full task; the second export time is the generation time of the full task.
[0140] S403. If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then execute the second task.
[0141] For example, the specific principles of steps S401-S403 in this embodiment can be found in [reference needed]. Figure 2 Steps S201-S203 in the above steps will not be described again here.
[0142] S404. If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then determine the completed incremental task corresponding to the data table indicated by the first task; the incremental task has a task sequence number, which is used to indicate the generation order of the incremental tasks among all the incremental tasks of the data table indicated by the incremental task.
[0143] For example, in this embodiment, the incremental task received by the electronic device has a task sequence number. For a data table, the task sequence number can be used to indicate the generation order of the incremental task in all incremental tasks corresponding to the data table. All incremental tasks corresponding to the data table are all incremental tasks generated by other electronic devices for the data table.
[0144] For example, when other electronic devices need to incrementally update data table A in an electronic device, they can generate incremental task 1, incremental task 2, incremental task 3, ..., incremental task n, respectively, with task numbers Za_01, Za_02, Za_03, ..., Za_0n (where n>3 and n is an integer), based on the generation order of the incremental tasks. Similarly, when other electronic devices need to incrementally update data table B in an electronic device, they can generate incremental task 1, incremental task 2, incremental task 3, ..., incremental task n, respectively, with task numbers Zb_01, Zb_02, Zb_03, ..., Zb_0n (where n>3 and n is an integer), based on the generation order of the incremental tasks.
[0145] When the electronic device determines that the data table indicated by the first task and the second task are different, it can first obtain the completed incremental task corresponding to the data table indicated by the first task.
[0146] In one example, when determining the completed incremental tasks corresponding to the data table indicated by the first task, the completed incremental tasks can be first filtered in the historical synchronization records of the electronic device, and then the completed incremental tasks corresponding to the data table can be filtered out from the completed incremental tasks.
[0147] In one example, step S404 may include the following steps:
[0148] In step S404, if it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the first task are determined to be the first task group.
[0149] In the second step of step S404, if it is determined that there is no full task in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0150] For example, in this embodiment, when executing step S404, if the electronic device determines that the data table indicated by the first task and the second task are different, then the completed full tasks corresponding to the data table can be determined first, and all the completed incremental tasks can be grouped into a first task group.
[0151] Then, by comparing the second export time of all full tasks in the first task group with the first export time of the first task, if it is determined that the second export time of all full tasks in the first task group is less than the first export time of the first task, the first task, i.e. the incremental task with the smallest first export time, can be executed directly.
[0152] It is understood that in this embodiment, by comparing the second export time of the completed full task corresponding to the data table with the first export time of the first task, the phenomenon of data table synchronization errors caused by continuing to execute incremental tasks before the second export time of the full task after the data table has already undergone full task processing can be avoided, thereby improving the accuracy of data table synchronization.
[0153] Building upon the above example, the method also includes:
[0154] If it is determined that there is a full task in the first task group whose second export time is greater than the first export time of the first task, then the status of the first task is set to the processed status.
[0155] For example, in this embodiment, by comparing the second export time of all tasks in the first task group with the first export time of the first task, if it is determined that the second export time of at least one full task in the first task group is greater than the first export time of the first task, it indicates that the data table has already undergone full task update processing. Therefore, there is no need to perform incremental tasks before the export time corresponding to the full task. The status of the first task is set to the processed state, thereby ensuring the accuracy of data table synchronization.
[0156] In one example, step S404, the second step, can be performed through the following steps:
[0157] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0158] For example, when performing the second step of step S404, in order to avoid the problem of inaccurate data table synchronization caused by multiple processes updating the same data table simultaneously in the electronic device, before determining the completed incremental task corresponding to the data table indicated by the first task, it can also be determined whether there is currently a process updating the data table. If there is no full task in the first task group with a second export time greater than the first export time of the first task, and neither the full task currently in the processing state nor the incremental task currently in the processing state contains the full task and incremental task corresponding to the data table indicated by the first task, it indicates that there is currently no process updating the data table indicated by the first task in the electronic device. At this time, the completed incremental task corresponding to the data table indicated by the first task can be determined.
[0159] If the full task currently in the processing state and the incremental task currently in the processing state contain the full task or incremental task corresponding to the data table indicated by the first task, then at this time, we can wait for the other processes to finish updating and processing before executing the first task, or repeat step S401.
[0160] It is understandable that, in this embodiment, in order to avoid the problem of inaccurate data table updates caused by multiple processes updating the same data table at the same time, this embodiment will also determine whether the full task and incremental task currently in the processing state contain the full task and incremental task corresponding to the data table indicated by the first task, thereby improving the accuracy of data updates during the data table synchronization update process.
[0161] S405. Based on the task sequence number of the first task, determine the previous task sequence number of the first task. The previous task sequence number is the task sequence number of the incremental task corresponding to the previous generation order of the first task among the generated incremental tasks in the corresponding data table.
[0162] For example, in this embodiment, after the first task is determined, the task number of the preceding task can be determined based on the task number in the first task. The preceding task is the incremental task preceding the first task in the generation order of the incremental tasks generated by the other electronic devices for the data table indicated by the first task.
[0163] For example, when other electronic devices are generating incremental tasks for data table A in the electronic device, they can generate incremental tasks sequentially, including task numbers Za_01, Za_02, Za_03, ..., Za_0n, based on the generation order of the incremental tasks, and then distribute these incremental tasks to the electronic device in sequence. If the task included in the first task needs to be Za_03, then the task number preceding that task number is Za_02.
[0164] S406. If it is determined that the completed incremental tasks include an incremental task with the previous task number, then execute the first task.
[0165] For example, if the completed incremental tasks corresponding to the data table indicated by the first task include an incremental task with a previous task number, it indicates that the data table has already executed the previous incremental task of the first task, and the first task can be executed directly at this time.
[0166] It is understandable that, in this embodiment, in order to avoid data errors in the data table during data synchronization due to the inconsistency between the order of received incremental tasks and the order of generation of incremental tasks caused by network jitter of electronic devices, this embodiment will determine whether the electronic device has executed the previous incremental task before executing the first task before executing the first task. The first task will be executed only when the previous incremental task has been completed, thereby improving the accuracy of data synchronization.
[0167] S407. If it is determined that the completed incremental tasks do not include incremental tasks with the previous task number, then repeat the step of determining the incremental task with the shortest first export time as the first task among the pending incremental tasks corresponding to each data table.
[0168] For example, if the completed incremental tasks corresponding to the data table indicated by the first task do not include an incremental task with the previous task number, it indicates that the data table has not yet executed the previous incremental task of the first task. In order to avoid the problem of inaccurate data table update results caused by directly executing the first task, step S401 can be repeated directly.
[0169] For example, if the preceding incremental task of the first task instructs to add a new row of data in the data table, and the first task instructs to modify a certain field in the newly added row of data, if the preceding incremental task of the first task is not executed before the first task is executed, and the first task is executed directly, the electronic device will report an error, and the data in the data table will be corrupted.
[0170] It is understood that in this embodiment, when it is determined that the previous incremental task of the first task has not been executed before the first task is executed, step S401 can be repeated so that after the first electronic device receives the previous incremental task of the first task, it can execute each incremental task in the order of generation of the incremental tasks, thereby ensuring the accuracy of data table synchronization.
[0171] In one example, step S407 can be achieved through the following steps:
[0172] In step S407, if it is determined that the completed incremental tasks do not include incremental tasks with the previous task number, then the current time is compared with the receiving time of the first task to obtain the time difference.
[0173] In the second step of step S407, if the time difference is less than or equal to the preset threshold, the step of determining the incremental task with the smallest first export time as the first task is repeated among the incremental tasks to be processed in each data table.
[0174] For example, in this embodiment, when the electronic device determines that the completed incremental tasks do not include an incremental task with a previous task number, it will also determine the difference between the current time and the time when the electronic device received the first task, i.e., the aforementioned time difference.
[0175] When the electronic device determines that the time difference is less than or equal to a preset threshold, the electronic device can repeat step S401. By repeating the above steps, the electronic device can receive the previous incremental task of the first task during the repetition process, so that the electronic device can perform incremental synchronization processing of the data table according to the generation order of the incremental tasks, thereby ensuring the accuracy of data table synchronization.
[0176] Based on the above example, the method further includes the following steps:
[0177] If the time difference is greater than the preset threshold, then the first task is executed.
[0178] For example, when the electronic device determines that the time difference is greater than a preset threshold, the electronic device believes that the previous task of the first task was lost during transmission. Therefore, in order to avoid the electronic device consuming too much time while waiting for the previous task of the first task, the first task can be executed directly.
[0179] It is understandable that in this embodiment, by setting a preset time threshold, when the difference between the time the electronic device receives the first task and the current time is greater than the preset time threshold, in order to avoid the electronic device consuming too much time while waiting for the previous task, the electronic device can directly execute the first task, thereby improving the synchronization efficiency of the data table.
[0180] In this embodiment, by comparing the second export time of the completed full task corresponding to the data table with the first export time of the first task, errors in data table synchronization caused by continuing to execute incremental tasks before the second export time of the full task after the data table has already undergone full task processing can be avoided, thus improving the accuracy of data table synchronization. To avoid inaccurate data table updates caused by multiple processes simultaneously updating the same data table, this embodiment also determines whether the currently processing full task and the currently processing incremental task contain the full task and incremental task corresponding to the data table indicated by the first task, thereby improving the accuracy of data updates during the data table synchronization update process.
[0181] Figure 5 This is a schematic diagram of the structure of a data table synchronization device provided in an embodiment of this application, as shown below. Figure 5 As shown, the data table synchronization device includes:
[0182] The first determining unit 51 is used to determine the incremental task with the smallest first export time as the first task among the incremental tasks to be processed corresponding to each data table; wherein, the incremental task is used to indicate that the data table corresponding to the incremental task is to be incrementally updated; the first export time is the generation time of the incremental task.
[0183] The second determining unit 52 is used to determine the full task with the longest second export time as the second task among the full tasks to be processed corresponding to each data table; wherein, the full task is used to indicate that the data table indicated by the full task is to be fully updated; the second export time is the generation time of the full task.
[0184] The first execution unit 53 is configured to execute the second task if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task.
[0185] The apparatus provided in this embodiment is used to implement the technical solution provided by the above method. Its implementation principle and technical effect are similar, and will not be described again.
[0186] Figure 6 This is a schematic diagram of the structure of the second type of data table synchronization device provided in the embodiments of this application. Figure 5 Based on the device structure shown, the first execution unit 53 in this embodiment includes:
[0187] The first determining module 531 is used to determine the completed full tasks corresponding to the data table indicated by the second task as the second task group if the data table indicated by the first task is the same as the data table indicated by the second task.
[0188] The first execution module 532 is used to execute the second task if it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task.
[0189] In some embodiments, the first execution module 532 is specifically used for:
[0190] If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the second task, then the second task is executed.
[0191] In some embodiments, the apparatus further includes:
[0192] The second execution unit 54 is used to execute the first task if it is determined that the data table indicated by the first task is different from the data table indicated by the second task.
[0193] In some embodiments, the incremental task has a task sequence number, which indicates the generation order of the incremental tasks among all incremental tasks in the data table indicated by the incremental task; the second execution unit 54 includes:
[0194] The second determining module 541 is used to determine the completed incremental task corresponding to the data table indicated by the first task if the data table indicated by the first task is different from the data table indicated by the second task.
[0195] The third determining module 542 is used to determine the previous task number of the first task based on the task number of the first task, wherein the previous task number is the task number of the incremental task corresponding to the previous generation order of the first task in the incremental tasks generated in the corresponding data table.
[0196] The second execution module 543 is used to execute the first task if it is determined that the completed incremental tasks include an incremental task with the previous task number.
[0197] In some embodiments, the apparatus further includes:
[0198] The third execution module 544 is used to repeatedly execute the step of determining the incremental task with the shortest first export time as the first task among the pending incremental tasks corresponding to each data table if it is determined that the completed incremental tasks do not include an incremental task with the previous task number.
[0199] In some embodiments, the third execution module 544 is specifically used for:
[0200] If it is determined that the completed incremental tasks do not include incremental tasks with the previous task number, then the current time is subtracted from the receiving time of the first task to obtain the time difference.
[0201] If the time difference is less than or equal to the preset threshold, the step of determining the incremental task with the smallest first export time as the first task is repeated among the incremental tasks to be processed in each data table.
[0202] In some embodiments, the third execution module 544 is further configured to:
[0203] If the time difference is greater than the preset threshold, then the first task is executed.
[0204] In some embodiments, the second determining module 541 is specifically used for:
[0205] If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the first task are determined to be the first task group.
[0206] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental tasks corresponding to the data table indicated by the first task are determined.
[0207] In some embodiments, the second determining module 541 is further configured to: if it is determined that there are full tasks in the first task group whose second export time is greater than the first export time of the first task, then set the status of the first task to the processed status.
[0208] In some embodiments, when the second determining module 541 performs the operation of determining the completed incremental task corresponding to the data table indicated by the first task if it is determined that there is no full task in the first task group whose second export time is greater than the first export time of the first task, the module is specifically used to:
[0209] If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
[0210] In some embodiments, the first task has data to be updated; the data table has a primary key field; the second execution unit 54 executes the first task, specifically for:
[0211] In the data to be updated in the first task, determine the update data corresponding to the primary key field of the data table indicated by the first task; if it is determined that the data table includes update data, replace the data in the row where the update data is located with the data to be updated; if it is determined that the data table does not include update data, add the data to be updated to the data table.
[0212] In some embodiments, the second task includes data to be updated; when the first execution unit 53 executes the second task, it is specifically used for:
[0213] Delete the data in the data table indicated by the second task, and write the data to be updated in the second task into the data table.
[0214] The apparatus provided in this embodiment is used to implement the technical solution provided by the above method. Its implementation principle and technical effect are similar, and will not be described again.
[0215] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 7 As shown, the electronic device includes:
[0216] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, memory 292, and communication interface 293 can communicate with each other via the bus 294. The communication interface 293 can be used for information transmission. The processor 291 can invoke logical instructions stored in the memory 292 to execute the methods of the above embodiments.
[0217] Furthermore, the logic instructions in the aforementioned memory 292 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.
[0218] The memory 292, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, thereby implementing the methods in the above-described method embodiments.
[0219] The memory 292 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 292 may include high-speed random access memory and may also include non-volatile memory.
[0220] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the methods.
[0221] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements any one of the methods.
[0222] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0223] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A data table synchronization method, characterized in that, include: Among the incremental tasks to be processed corresponding to each data table, the incremental task with the shortest first export time is determined as the first task; wherein, the incremental task is used to indicate that incremental update processing is performed on the data table corresponding to the incremental task; the first export time is the generation time of the incremental task; Among the pending full tasks corresponding to each data table, the full task with the longest second export time is determined as the second task; wherein, the full task is used to instruct a full update process for the data table indicated by the full task; the second export time is the generation time of the full task; If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the second task is executed; if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the second task is executed, including: If it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the second task are determined to be the second task group. If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, then the second task is executed.
2. The method according to claim 1, characterized in that, If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, then the second task is executed, including: If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the second task, then the second task is executed.
3. The method according to claim 1, characterized in that, The method further includes: If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the first task is executed.
4. The method according to claim 3, characterized in that, The incremental task has a task sequence number, which is used to indicate the generation order of the incremental task among all the incremental tasks in the data table indicated by the incremental task. If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the first task is executed, including: If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed incremental task corresponding to the data table indicated by the first task is determined. Based on the task number of the first task, determine the previous task number of the task number of the first task, wherein the previous task number is the task number of the incremental task corresponding to the previous generation order of the generation order of the first task in the generated incremental tasks corresponding to the data table. If it is determined that the completed incremental tasks include an incremental task with the previous task number, then the first task is executed.
5. The method according to claim 4, characterized in that, The method further includes: If it is determined that the completed incremental tasks do not include an incremental task with the previous task number, then the step of determining the incremental task with the shortest first export time as the first task among the pending incremental tasks corresponding to each data table is repeated.
6. The method according to claim 5, characterized in that, If it is determined that the completed incremental tasks do not include an incremental task with the previous task number, then the step of determining the incremental task with the shortest first export time as the first task from the pending incremental tasks corresponding to each data table is repeated, including: If it is determined that the completed incremental tasks do not include incremental tasks with the previous task sequence number, then the current time is subtracted from the receiving time of the first task to obtain the time difference value. If the time difference is less than or equal to a preset threshold, then the step of determining the incremental task with the smallest first export time as the first task among the incremental tasks to be processed in each data table is repeated.
7. The method according to claim 6, characterized in that, The method further includes: If the time difference is greater than the preset threshold, then the first task is executed.
8. The method according to claim 4, characterized in that, If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed incremental tasks corresponding to the data table indicated by the first task are determined, including: If it is determined that the data table indicated by the first task is different from the data table indicated by the second task, then the completed full tasks corresponding to the data table indicated by the first task are determined to be the first task group. If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
9. The method according to claim 8, characterized in that, The method further includes: If it is determined that there is a full set of tasks in the first task group whose second export time is greater than the first export time of the first task, then the status of the first task is set to the processed status.
10. The method according to claim 8, characterized in that, If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, then the completed incremental tasks corresponding to the data table indicated by the first task are determined, including: If it is determined that there are no full tasks in the first task group whose second export time is greater than the first export time of the first task, and neither the full tasks currently in the processing state nor the incremental tasks currently in the processing state contain the full tasks and incremental tasks corresponding to the data table indicated by the first task, then the completed incremental task corresponding to the data table indicated by the first task is determined.
11. The method according to any one of claims 1-10, characterized in that, The first task has data to be updated; the data table has a primary key field; executing the first task includes: In the data to be updated in the first task, determine the update data corresponding to the primary key field of the data table indicated by the first task; If it is determined that the data table includes the updated data, then the data in the row containing the updated data in the data table is replaced with the data to be updated; If it is determined that the data table does not contain the updated data, then the data to be updated is added to the data table.
12. The method according to any one of claims 1-10, characterized in that, The second task includes the data to be updated; performing the second task includes: Delete the data in the data table indicated by the second task, and write the data to be updated in the second task into the data table.
13. A data table synchronization device, characterized in that, include: The first determining unit is used to determine the incremental task with the shortest first export time as the first task among the incremental tasks to be processed corresponding to each data table; wherein, the incremental task is used to indicate that incremental update processing is performed on the data table corresponding to the incremental task; the first export time is the generation time of the incremental task; The second determining unit is used to determine the full task with the longest second export time as the second task among the full tasks to be processed corresponding to each data table; wherein, the full task is used to instruct a full update process for the data table indicated by the full task; and the second export time is the generation time of the full task. The first execution unit is configured to execute the second task if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task. The first execution unit is specifically configured to: if it is determined that the data table indicated by the first task is the same as the data table indicated by the second task, then determine that the completed full tasks corresponding to the data table indicated by the second task are the second task group; If it is determined that there are no full tasks in the second task group whose second export time is greater than the second export time of the second task, then the second task is executed.
14. An electronic device comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-12.
Citation Information
Patent Citations
Method and device for data migration
CN107391634A
Timestamp-based data synchronization method and device, and computer equipment
CN111245548A