Data synchronization method, device and equipment, medium and program product

Abstract

The invention discloses a data synchronization method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining the change information of a current data change of a preset original data table when the data change of the original data table is detected; generating a change record according to the change information, and adding the change record into a preset change record table; and when a data paging synchronization request sent by a service system is received, sending the change records in the change record table to the service system in a paging manner. According to the method and the device, the data paging synchronization is realized by utilizing the change record table which is associated with the original data table and can only add the change record, so that the problem of data omission in the data synchronization process can be effectively avoided even under the condition that the data of the data source is frequently changed; and the data synchronized to the service system is consistent with the data of the data source.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data synchronization method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] At present, there are two main ways to synchronize data between two systems, depending on the initiator of the synchronization request: one is that the data source system actively initiates synchronization, and pushes the data source change data to the target business system through message push or interface push. The target business system then changes the local data according to the change message to achieve data synchronization; the other is that the target business system actively initiates synchronization, pulls data through the API (Application Programming Interface) provided by the data source system, and then traverses and compares the local data one by one to achieve data synchronization; according to the different API functions provided by the data source system, the data processing method is different. Generally, the data source system mainly provides the following API functions: API for one-time query of all data, API for paging query of all data, API for one-time query of data according to data growth ID or data change time, API for paging query of data according to data growth ID or data change time.

[0003] For data synchronization methods based on message push or interface push, when data synchronization and integration is performed on a third-party system, the system itself needs to be modified each time to configure the push address of the third-party system. For data synchronization methods based on API, when data synchronization and integration is performed on a third-party system, the third-party system only needs to pull data according to the API and synchronize the data, and the system itself has better maintainability. Among them, the data synchronization method of querying all data at one time is only applicable to data sources with a small amount of data. For data sources with a large amount of data or data sources with a growing amount of data, it is advisable to use a paging query method for data synchronization. However, the paging query data APIs provided by the data source in the prior art are all paging data query APIs based on the data source table. However, when the data source data changes frequently, the paging query synchronization may cause data omissions, resulting in inconsistencies between the data synchronized to the target business system and the data source data. Summary of the invention

[0004] The present invention provides a data synchronization method, apparatus, equipment, medium and program product, which realizes data paging synchronization by utilizing a change record table associated with an original data table and to which only new change records can be added. Even when the data source data changes frequently, the problem of data omission in the data synchronization process can be effectively avoided, thereby ensuring that the data synchronized to the business system is consistent with the data source data.

[0005] In order to solve the above technical problems, a first aspect of an embodiment of the present invention provides a data synchronization method, which is executed by a data source system and includes:

[0006] When a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change;

[0007] Generate a change record based on the change information and add it to a preset change record table;

[0008] When a data paging synchronization request sent by the business system is received, the change record paging in the change record table is sent to the business system.

[0009] As a preferred solution, when detecting that a data change occurs in a preset original data table, obtaining the change information of the original data table at the time of the data change specifically includes:

[0010] When a data change in the original data table is detected, the change type, change time and business information corresponding to the changed data in the original data table are obtained;

[0011] According to the change type, change time and business information corresponding to the data changed at that time, the change information of the original data table during the data change at that time is determined.

[0012] As a preferred solution, determining the change information of the original data table in the current data change according to the change type, change time and business information corresponding to the data changed at that time specifically includes:

[0013] Determine the change event identifier corresponding to the data that was changed in the original data table according to the change time and the preset change event identifier corresponding to each change record in the change record table;

[0014] According to the change type, business information and change event identifier corresponding to the data changed at that time, the change information of the original data table at that time of data change is determined.

[0015] As a preferred solution, each change record in the change record table is sorted in ascending order according to the change event identifier, so that the change record with an earlier change time is sorted higher in the change record table.

[0016] As a preferred solution, when receiving a data paging synchronization request sent by a business system, sending the change record paging in the change record table to the business system specifically includes:

[0017] When receiving a data paging synchronization request sent by the business system, obtaining a request query page number and the amount of query data per page corresponding to the data paging synchronization request;

[0018] In response to the data paging synchronization request, a number of target change records corresponding to the requested query page number are obtained from the change record table; wherein the number of the target change records is equal to or less than the number of query data per page;

[0019] Sending a plurality of the target change records to the business system.

[0020] A second aspect of an embodiment of the present invention provides a data synchronization method, which is executed by a business system and includes:

[0021] Generate a data paging synchronization request at the current moment and send it to the data source system according to the preset request query page number and the number of query data per page; wherein the data paging synchronization request is used to instruct the data source system to send the target change record corresponding to the request query page number;

[0022] When receiving a plurality of target change records corresponding to the requested query page number sent by the data source system, updating the preset local data table based on the target change records; wherein the target change records are generated by the data source system by executing the data synchronization method as described in any one of the first aspects;

[0023] The request query page number is increased by 1 to update the request query page number, and the aforementioned steps are re-executed based on the updated request query page number until no target change record corresponding to the updated request query page number sent by the data source system is received.

[0024] As a preferred solution, the target change record includes the change type, business information and change event identifier corresponding to the changed data; the business information includes the business identifier and business attributes;

[0025] Then, updating the preset local data table based on the target change record specifically includes:

[0026] Obtaining the change type recorded in each of the target change records;

[0027] When the change type of any target change record is a new addition type, obtaining the business information recorded in the any target change record and adding it to the local data table;

[0028] When the change type of any target change record is a modification type, obtaining a service identifier and a service attribute recorded in the any target change record;

[0029] Using the business attributes recorded in any one of the target change records, the business attributes of the data matching the business identifier in the local data table are updated;

[0030] When the change type of any target change record is a deletion type, obtaining a service identifier recorded in the any target change record;

[0031] The data matching the service identifier in the local data table is deleted.

[0032] A third aspect of an embodiment of the present invention provides a data synchronization device, including:

[0033] Change information acquisition module, used to:

[0034] When a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change;

[0035] Change record generation module, used to:

[0036] Generate a change record based on the change information and add it to a preset change record table;

[0037] Change record sending module, used for:

[0038] When a data paging synchronization request sent by the business system is received, the change record paging in the change record table is sent to the business system.

[0039] A fourth aspect of an embodiment of the present invention provides a data synchronization device, including:

[0040] Data synchronization request module, used to:

[0041] Generate a data paging synchronization request at the current moment and send it to the data source system according to the preset request query page number and the number of query data per page; wherein the data paging synchronization request is used to instruct the data source system to send the target change record corresponding to the request query page number;

[0042] Data synchronization module, used for:

[0043] When receiving a plurality of target change records corresponding to the requested query page number sent by the data source system, updating the preset local data table based on the target change records; wherein the target change records are generated by the data source system by executing the data synchronization method as described in any one of the first aspects;

[0044] The request query page number is increased by 1 to update the request query page number, and the aforementioned steps are re-executed based on the updated request query page number until no target change record corresponding to the updated request query page number sent by the data source system is received.

[0045] A fifth aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the data synchronization method described in any one of the first aspect, or implements the data synchronization method described in any one of the second aspect.

[0046] A sixth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the data synchronization method described in any one of the first aspect, or execute the data synchronization method described in any one of the second aspect.

[0047] A seventh aspect of an embodiment of the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the data synchronization method described in any one of the first aspect, or implements the steps of the data synchronization method described in any one of the second aspect.

[0048] Compared with the prior art, the beneficial effect of the embodiments of the present invention is that data paging synchronization is achieved by utilizing a change record table that is associated with the original data table and to which only new change records can be added. Even when the data source data changes frequently, the problem of data omission during the data synchronization process can be effectively avoided, thereby ensuring that the data synchronized to the business system is consistent with the data source data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a data synchronization method performed by a data source system in an embodiment of the present invention;

[0050] Figure 2 is a flow chart of change record generation and insertion in an embodiment of the present invention;

[0051] Figure 3 This is a system architecture diagram for performing data paging synchronization between a data source system and a business system in an embodiment of the present invention;

[0052] Figure 4 is a flowchart of a data synchronization method executed by a business system in an embodiment of the present invention;

[0053] Figure 5 is a flow chart of performing data paging synchronization based on change records in an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of the structure of a data synchronization device applied to a data source system in an embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of the structure of a data synchronization device applied to a business system in an embodiment of the present invention;

[0056] Figure 8 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] See also Figure 1 According to a first aspect of an embodiment of the present invention, a data synchronization method is provided. The method is executed by a data source system and includes the following steps S101 to S103:

[0059] Step S101, when it is detected that a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change;

[0060] Step S102, generating a change record according to the change information and adding it to a preset change record table;

[0061] Step S103, when a data paging synchronization request sent by the business system is received, the change record paging in the change record table is sent to the business system.

[0062] It is worth mentioning that the reasons why the paging query synchronization method in the prior art is prone to data omissions are as follows: assuming that the original data table in the data source system undergoes data changes as the business is used, such as addition, modification, deletion, etc., each change requires updating the latest modification time of the data in the original data table, and the data synchronization task only needs to sort the data in ascending order according to the latest modification time, and then paging read the data in the original data table whose latest modification time is after the last synchronization time, so as to achieve synchronization of changed data between two data synchronization tasks and improve the efficiency of data synchronization. Assuming that the data in the original data table has not changed during the data synchronization task, the business system sets the number of data to be queried per page of the data synchronization task to 100 data items, and starts querying and synchronizing from the first page of data. The first query is for data items 1 to 100 on the first page, the second query is for data items 101 to 200 on the second page, and the nth query is for data items 100*(n-1)+1 to 100n on the nth page, until the mth query finds that the data on the mth page is less than 100, indicating that all data in the original data table has been synchronized. At this time, the business system records the synchronization time and ends this data synchronization task. In this way, you only need to set a periodic scheduled data synchronization task in the business system to ensure that the data in the business system remains consistent with the data source system within a cycle.

[0063] However, the above data synchronization process assumes that the data in the original data table has not changed during the data synchronization task. In actual situations, if the amount of data to be synchronized in a data synchronization task is large, the number of paging queries is large, and the business of the data source system occurs frequently, it is easy for the data in the original data table to change during the data synchronization task. For example, when querying data by paging for the nth time, due to business updates, a certain data before the nth page has changed, and the latest modification time of the data will become the current time, causing the data to be sorted at the end of the original data table. At this time, the sorting of the data in the queried nth page will shift forward, that is, the 100*(n-1)+1th data that has not yet been synchronized will become the 100*(n-1)th data, and the data will be missed when the nth paging query is performed.

[0064] In order to solve the problems existing in the above-mentioned prior art, this embodiment sets a corresponding change record table for the original data table to be synchronized, such as Figure 2As shown in the figure, it is a flowchart of change record generation and insertion. When data changes are detected in the original data table, such as addition, modification and deletion, the change information of the original data table at that time is obtained, and then the change record is generated and added to the change record table. It is worth noting that the change record table records the latest data information of the source record each time a change occurs, that is, the change information. Since the change record is used to record the change information of each change of any data in the original data table, the change record only supports adding to the change record table and cannot be modified or deleted. Therefore, during the data synchronization process with the business system, the situation of "the data before the paging data being synchronized has changed" will not occur, thereby avoiding the problem of data omission during the data synchronization process.

[0065] Furthermore, this embodiment provides a paging query API for the change record table for the business system, such as Figure 3 The figure shows a system architecture diagram for data paging synchronization between a data source system and a business system. When a data paging synchronization request is received from a business system, the change record pages in the change record table are sent to the business system in response to the requested query page number and the amount of query data per page indicated by the data paging synchronization request, so that the business system can perform data synchronization based on the change information of the data recorded in the received change record.

[0066] The data synchronization method provided in an embodiment of the present invention realizes data paging synchronization by utilizing a change record table that is associated with the original data table and to which only new change records can be added. Even when the data source data changes frequently, it can effectively avoid the problem of data omission during the data synchronization process and ensure that the data synchronized to the business system is consistent with the data source data.

[0067] As a preferred solution, when detecting that a data change occurs in a preset original data table, obtaining the change information of the original data table at the time of the data change specifically includes:

[0068] When a data change in the original data table is detected, the change type, change time and business information corresponding to the changed data in the original data table are obtained;

[0069] According to the change type, change time and business information corresponding to the data changed at that time, the change information of the original data table during the data change at that time is determined.

[0070] It is worth noting that in order to ensure the accuracy of data synchronization, when the present embodiment detects that a data change has occurred in the original data table, it obtains the change type and business information corresponding to the data that was changed in the original data table, thereby ensuring that after receiving the change record, the business system can clarify the currently required data synchronization operation, such as adding data, modifying data, or deleting data, and can clarify the specific content of data synchronization. For example, when new data needs to be added, the business information of the newly added data needs to be clarified; when data needs to be modified, the current modification object and modification content need to be clarified based on the business information; when data needs to be deleted, the current deletion object needs to be clarified based on the business information. This embodiment will not be described in detail here. It is worth noting that the original record table records the latest business information of each source data, while the business information recorded in the change record table is the latest business information of each source data at each change. In addition, considering that data paging synchronization usually starts from the homepage to traverse the data, and then selects the data with the latest modification time after the last synchronization time, in order to ensure that the values ​​of each data are the latest values ​​after the business system completes data synchronization, this embodiment also obtains the change time corresponding to the data that was changed at that time, so that the change records in the change record table can be sorted in ascending order based on the change time, ensuring that the change record of the latest changed data is sorted last in the change record table.

[0071] As a preferred solution, determining the change information of the original data table in the current data change according to the change type, change time and business information corresponding to the data changed at that time specifically includes:

[0072] Determine the change event identifier corresponding to the data changed at that time in the original data table according to the change time and the preset change event identifier corresponding to each change record in the change record table;

[0073] According to the change type, business information and change event identifier corresponding to the data changed at that time, the change information of the original data table at that time of data change is determined.

[0074] Specifically, this embodiment uses a change event identifier to characterize the change time of each change record. It can be understood that the change time corresponding to the data that was changed should be the latest change time. In order to confirm its change event identifier, it is necessary to consider the change event identifier corresponding to each change record in the current change record table, so that the change event identifier corresponding to the data that was changed is larger than the change event identifier corresponding to each change record in the current change record table. For example, the change event identifier is represented in digital form. Assuming that there are three change records in the current change record table, the change event identifier of the first change record is 1, the change event identifier of the second change record is 2, and the change event identifier of the third change record is 3, then the change event identifier corresponding to the data that was changed can be 4, 5, 8, 10, 11, etc. It is only necessary to ensure that the change event identifier corresponding to the data that was changed is larger than the change event identifier of the previously added change record. This embodiment is not specifically limited here. In addition, the change event identifier in this embodiment is not limited to being represented in digital form, and can also be a combination of English and numbers, such as A1, A2, A3, A4, etc. This embodiment is not specifically limited here.

[0075] As a preferred solution, each change record in the change record table is sorted in ascending order according to the change event identifier, so that the change record with an earlier change time is sorted higher in the change record table.

[0076] Specifically, since the change event identifier has the characteristic that the later the change record is added, the larger the change event identifier is, this embodiment can ensure that the change record of the latest changed data is sorted last in the change record table by sorting the change records in the change record table in ascending order according to the change event identifier, thereby ensuring that after the business system completes data synchronization, the values ​​of each data are the latest values.

[0077] As a preferred solution, when receiving a data paging synchronization request sent by a business system, sending the change record paging in the change record table to the business system specifically includes:

[0078] When receiving a data paging synchronization request sent by the business system, obtaining a request query page number and the amount of query data per page corresponding to the data paging synchronization request;

[0079] In response to the data paging synchronization request, a number of target change records corresponding to the requested query page number are obtained from the change record table; wherein the number of the target change records is equal to or less than the number of query data per page;

[0080] Sending a plurality of the target change records to the business system.

[0081] It is worth noting that the data paging synchronization request includes the requested query page number and the number of query data per page set by the business system. For example, the requested query page number can be page 1, page 2, page 4, page 6, etc., and the number of query data per page can be 10, 25, 48, 65, 80, 100, etc. This embodiment will not be described in detail here. Based on the number of query data per page, all change records in the change record table are paged to clarify the change records contained in each page. Further, a number of target change records corresponding to the requested query page number are obtained from the change record table. It can be understood that when the requested query page number is the last page of the change record table, the number of target change records may be equal to the number of query data per page, or it may be less than the number of query data per page. When the requested query page number is not the last page of the change record table, the number of target change records must be equal to the number of query data per page.

[0082] See also Figure 4 A second aspect of an embodiment of the present invention provides a data synchronization method, which is executed by a business system and includes the following steps S201 to S203:

[0083] Step S201, generating a data paging synchronization request at the current moment and sending it to the data source system according to the preset request query page number and the number of query data per page; wherein the data paging synchronization request is used to instruct the data source system to send a target change record corresponding to the request query page number;

[0084] Step S202: when receiving a plurality of target change records corresponding to the requested query page number sent by the data source system, updating a preset local data table based on the target change records; wherein the target change records are generated by the data source system by executing the data synchronization method as described in any one of the first aspects;

[0085] Step S203, adding 1 to the request query page number to update the request query page number, and re-executing the above steps based on the updated request query page number until no target change record corresponding to the updated request query page number sent by the data source system is received.

[0086] Specifically, based on the currently preset request query page number, such as page 1, page 3, page 5, etc., and the number of query data per page, such as 10, 25, 48, 65, 80, 100, etc., a corresponding data paging synchronization request is generated and sent to the data source system. It is worth noting that in order to ensure that all data in the local data table are synchronized to the latest value, the preset request query page number can be set to page 1; in addition, since the change record is used to record the change information of each change of any data in the original data table, the change record only supports adding to the change record table, and cannot be modified or deleted. Therefore, the initial request query page number for each data synchronization can be determined based on the paging query page number when the last data synchronization was completed. For example, when the paging query page number when the last data synchronization was completed is page 3, it means that the change records from page 1 to page 3 in the change record table have been synchronized. The initial request query page number for this data synchronization can be set to page 4.

[0087] Furthermore, if Figure 5 As shown, when several target change records corresponding to the requested query page number are received from the data source system, in order to ensure that the values ​​of each data are the latest values ​​after data synchronization is completed, each target change record is traversed in ascending order according to the change time or change event identifier, and then the local data table is updated based on each target change record. After the traversal of each current target change record is completed, the current requested query page number is increased by 1, indicating that the data corresponding to the current requested query page number has been synchronized and the data of the next page needs to be synchronized. Therefore, steps S201 to S202 are re-executed based on the updated requested query page number until the target change record corresponding to the updated requested query page number sent by the data source system is not received, indicating that all data in the change record table in the data source system has been synchronized.

[0088] The data synchronization method provided in an embodiment of the present invention realizes data paging synchronization by utilizing a change record table that is associated with the original data table and to which only new change records can be added. Even when the data source data changes frequently, it can effectively avoid the problem of data omission during the data synchronization process and ensure that the data synchronized to the business system is consistent with the data source data.

[0089] As a preferred solution, the target change record includes the change type, business information and change event identifier corresponding to the changed data; the business information includes the business identifier and business attributes;

[0090] Then, updating the preset local data table based on the target change record specifically includes:

[0091] Obtaining the change type recorded in each of the target change records;

[0092] When the change type of any target change record is a new addition type, obtaining the business information recorded in the any target change record and adding it to the local data table;

[0093] When the change type of any target change record is a modification type, obtaining a service identifier and a service attribute recorded in the any target change record;

[0094] Using the business attributes recorded in any one of the target change records, the business attributes of the data matching the business identifier in the local data table are updated;

[0095] When the change type of any target change record is a deletion type, obtaining a service identifier recorded in the any target change record;

[0096] The data matching the service identifier in the local data table is deleted.

[0097] It is worth noting that the fields included in the change record are as follows: change event identifier, change type, business identifier, business attribute 1, business attribute 2, ..., business attribute n; the fields included in each data in the original data table are as follows: business identifier, business attribute 1, business attribute 2, ..., business attribute n. When the change type of any target change record is a new type, the business information recorded in the target change record is obtained, including the business identifier and all the corresponding business attributes, and then the business information is added to the local data table; when the change type of any target change record is a modification type, since the business identifier corresponding to each data is unique, the matching data to be synchronized in the local data table can be identified based on the business identifier, and the business attributes of the data to be synchronized are updated to the business attributes recorded in the target change record; when the change type of any target change record is a deletion type, it is only necessary to determine the matching data to be deleted in the local data table based on the business identifier recorded in the target change record and delete it.

[0098] See also Figure 6 A third aspect of an embodiment of the present invention provides a data synchronization device 100 applied to a data source system, comprising:

[0099] The change information acquisition module 11 is used to:

[0100] When a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change;

[0101] The change record generation module 12 is used to:

[0102] Generate a change record based on the change information and add it to a preset change record table;

[0103] The change record sending module 13 is used to:

[0104] When a data paging synchronization request sent by the business system is received, the change record paging in the change record table is sent to the business system.

[0105] As a preferred solution, the change information acquisition module 11 is used to acquire the change information of the original data table at the current data change when detecting that the preset original data table has a data change, specifically including:

[0106] When a data change in the original data table is detected, the change type, change time and business information corresponding to the changed data in the original data table are obtained;

[0107] According to the change type, change time and business information corresponding to the data changed at that time, the change information of the original data table during the data change at that time is determined.

[0108] As a preferred solution, the change information acquisition module 11 is used to determine the change information of the original data table in the current data change according to the change type, change time and business information corresponding to the data changed at that time, specifically including:

[0109] Determine the change event identifier corresponding to the data changed at that time in the original data table according to the change time and the preset change event identifier corresponding to each change record in the change record table;

[0110] According to the change type, business information and change event identifier corresponding to the data changed at that time, the change information of the original data table at that time of data change is determined.

[0111] As a preferred solution, each change record in the change record table is sorted in ascending order according to the change event identifier, so that the change record with an earlier change time is sorted higher in the change record table.

[0112] As a preferred solution, the change record sending module 13 is used to send the change record pages in the change record table to the business system when receiving a data page synchronization request sent by the business system, specifically including:

[0113] When receiving a data paging synchronization request sent by the business system, obtaining a request query page number and the amount of query data per page corresponding to the data paging synchronization request;

[0114] In response to the data paging synchronization request, a number of target change records corresponding to the requested query page number are obtained from the change record table; wherein the number of the target change records is equal to or less than the number of query data per page;

[0115] Sending a plurality of the target change records to the business system.

[0116] The data synchronization device 100 applied to the data source system provided by the embodiment of the present invention realizes data paging synchronization by utilizing a change record table associated with the original data table and to which only new change records can be added. Even when the data source data changes frequently, it can effectively avoid the problem of data omission during the data synchronization process and ensure that the data synchronized to the business system is consistent with the data source data.

[0117] See also Figure 7 According to a fourth aspect of an embodiment of the present invention, there is provided a data synchronization device 200 applied to a business system, comprising:

[0118] The data synchronization request module 21 is used to:

[0119] Generate a data paging synchronization request at the current moment and send it to the data source system according to the preset request query page number and the number of query data per page; wherein the data paging synchronization request is used to instruct the data source system to send the target change record corresponding to the request query page number;

[0120] The data synchronization module 22 is used for:

[0121] When receiving a plurality of target change records corresponding to the requested query page number sent by the data source system, updating the preset local data table based on the target change records; wherein the target change records are generated by the data source system by executing the data synchronization method as described in any embodiment of the first aspect;

[0122] The request query page number is increased by 1 to update the request query page number, and the aforementioned steps are re-executed based on the updated request query page number until no target change record corresponding to the updated request query page number sent by the data source system is received.

[0123] As a preferred solution, the target change record includes the change type, business information and change event identifier corresponding to the changed data; the business information includes the business identifier and business attributes;

[0124] Then, the data synchronization module 22 is used to update the preset local data table based on the target change record, specifically including:

[0125] Obtaining the change type recorded in each of the target change records;

[0126] When the change type of any target change record is a new addition type, obtaining the business information recorded in the any target change record and adding it to the local data table;

[0127] When the change type of any target change record is a modification type, obtaining a service identifier and a service attribute recorded in the any target change record;

[0128] Using the business attributes recorded in any one of the target change records, the business attributes of the data matching the business identifier in the local data table are updated;

[0129] When the change type of any target change record is a deletion type, obtaining a service identifier recorded in the any target change record;

[0130] The data matching the service identifier in the local data table is deleted.

[0131] The data synchronization device 200 applied to the business system provided by the embodiment of the present invention realizes data paging synchronization by utilizing a change record table associated with the original data table and to which only new change records can be added. Even when the data source data changes frequently, it can effectively avoid the problem of data omission during the data synchronization process and ensure that the data synchronized to the business system is consistent with the data source data.

[0132] See also Figure 8 A fifth aspect of an embodiment of the present invention provides an electronic device 300, comprising a memory 32, a processor 31, and a computer program stored in the memory 32 and executable on the processor 31, wherein when the processor 31 executes the computer program, the data synchronization method described in any embodiment of the first aspect is implemented, or the data synchronization method described in any embodiment of the second aspect is implemented.

[0133] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device 300.

[0134] The electronic device 300 may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 300 and does not constitute a limitation on the electronic device 300. The electronic device 300 may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the electronic device 300 may also include input and output devices, network access devices, buses, etc.

[0135] The processor 31 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 31 may also be any conventional processor 31, etc. The processor 31 is the control center of the electronic device 300, and uses various interfaces and lines to connect various parts of the entire electronic device 300.

[0136] The memory 32 can be used to store the computer program and / or module. The processor 31 implements various functions of the electronic device 300 by running or executing the computer program and / or module stored in the memory 32 and calling the data stored in the memory 32. The memory 32 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0137] A sixth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the data synchronization method described in any embodiment of the first aspect, or to execute the data synchronization method described in any embodiment of the second aspect.

[0138] A seventh aspect of an embodiment of the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the data synchronization method described in any embodiment of the first aspect, or implements the steps of the data synchronization method described in any embodiment of the second aspect.

[0139] Wherein, if the module / unit integrated in the electronic device 300 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0140] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A data synchronization method, characterized in that: The method is executed by a data source system and includes: When a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change; Generate a change record based on the change information and add it to a preset change record table; When a data paging synchronization request sent by the business system is received, the change record paging in the change record table is sent to the business system.

2. The data synchronization method according to claim 1, characterized in that: When detecting that a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change specifically includes: When a data change in the original data table is detected, the change type, change time and business information corresponding to the changed data in the original data table are obtained; According to the change type, change time and business information corresponding to the data changed at that time, the change information of the original data table during the data change at that time is determined.

3. The data synchronization method according to claim 2, characterized in that: The determining, according to the change type, change time and business information corresponding to the data changed at that time, the change information of the original data table at that time of data change specifically includes: Determine the change event identifier corresponding to the data that was changed in the original data table according to the change time and the preset change event identifier corresponding to each change record in the change record table; According to the change type, business information and change event identifier corresponding to the data changed at that time, the change information of the original data table at that time of data change is determined.

4. The data synchronization method according to claim 3, characterized in that: The change records in the change record table are sorted in ascending order according to the change event identifier, so that the change records with earlier change times are sorted higher in the change record table.

5. The data synchronization method according to claim 1, characterized in that: When receiving a data paging synchronization request sent by the business system, sending the change record paging in the change record table to the business system specifically includes: When receiving a data paging synchronization request sent by the business system, obtaining a request query page number and the amount of query data per page corresponding to the data paging synchronization request; In response to the data paging synchronization request, a number of target change records corresponding to the requested query page number are obtained from the change record table; wherein the number of the target change records is equal to or less than the number of query data per page; Sending a plurality of the target change records to the business system.

6. A data synchronization method, characterized in that: The method is executed by a business system and includes: Generate a data paging synchronization request at the current moment and send it to the data source system according to the preset request query page number and the number of query data per page; wherein the data paging synchronization request is used to instruct the data source system to send the target change record corresponding to the request query page number; When receiving a plurality of target change records corresponding to the requested query page number sent by the data source system, updating the preset local data table based on the target change records; wherein the target change records are generated by the data source system by executing the data synchronization method according to any one of claims 1 to 5; The request query page number is increased by 1 to update the request query page number, and the aforementioned steps are re-executed based on the updated request query page number until no target change record corresponding to the updated request query page number sent by the data source system is received.

7. The data synchronization method according to claim 6, characterized in that: The target change record includes the change type, business information and change event identifier corresponding to the changed data; the business information includes the business identifier and business attributes; Then, updating the preset local data table based on the target change record specifically includes: Obtaining the change type recorded in each of the target change records; When the change type of any target change record is a new addition type, obtaining the business information recorded in the any target change record and adding it to the local data table; When the change type of any target change record is a modification type, obtaining a service identifier and a service attribute recorded in the any target change record; Using the business attributes recorded in any one of the target change records, the business attributes of the data matching the business identifier in the local data table are updated; When the change type of any target change record is a deletion type, obtaining a service identifier recorded in the any target change record; The data matching the service identifier in the local data table is deleted.

8. A data synchronization device, characterized in that: include: Change information acquisition module, used to: When a data change occurs in a preset original data table, obtaining change information of the original data table at the time of the data change; Change record generation module, used to: Generate a change record based on the change information and add it to a preset change record table; Change record sending module, used for: When a data paging synchronization request sent by the business system is received, the change record paging in the change record table is sent to the business system.

9. A data synchronization device, characterized in that: include: Data synchronization request module, used to: Generate a data paging synchronization request at the current moment and send it to the data source system according to the preset request query page number and the number of query data per page; wherein the data paging synchronization request is used to instruct the data source system to send the target change record corresponding to the request query page number; Data synchronization module, used for: When receiving a plurality of target change records corresponding to the requested query page number sent by the data source system, updating the preset local data table based on the target change records; wherein the target change records are generated by the data source system by executing the data synchronization method according to any one of claims 1 to 5; The request query page number is increased by 1 to update the request query page number, and the aforementioned steps are re-executed based on the updated request query page number until no target change record corresponding to the updated request query page number sent by the data source system is received.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the data synchronization method described in any one of claims 1 to 5 or the data synchronization method described in any one of claims 6 to 7 when executing the computer program.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the data synchronization method described in any one of claims 1 to 5, or to execute the data synchronization method described in any one of claims 6 to 7.

12. A computer program product, characterized in that It comprises a computer program / instruction, which, when executed by a processor, implements the steps of the data synchronization method described in any one of claims 1 to 5, or implements the steps of the data synchronization method described in any one of claims 6 to 7.

Images