Data incremental synchronization method, device, equipment and storage medium

Abstract

The present invention provides a data incremental synchronization method, device, equipment and storage medium, which can be applied to the fields of computer technology and database technology. The method includes: in response to a data change in a target data table stored in a target database cluster, determining a to-be-updated cluster of the target database cluster based on a cluster topology diagram; determining a to-be-updated data structure of the to-be-updated data table stored in the to-be-updated cluster, a target data structure of the target data table and an update range of the to-be-updated data table; updating the to-be-updated data structure based on the target data structure and the to-be-updated data structure to obtain an updated data structure; determining an incremental synchronization statement corresponding to each data partition in the updated data structure based on the target data table, the to-be-updated data table and the update range; and executing incremental synchronization statements corresponding to the data partitions on multiple data partitions of the to-be-updated data table, respectively, to achieve data incremental synchronization.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and database technology, and in particular to a data incremental synchronization method, device, equipment and storage medium. Background Art

[0002] In the era of big data, the amount of data and the importance of the information stored in the data are growing rapidly. The increase in data volume has led to more complex dependencies between different data tables, and the increase in the importance of the information stored in the data has led to the need for more complete data backup. In related technologies, in order to ensure the correctness of the backup data, data backup is usually performed by backing up all the data in the main table to the secondary table after the main table is updated.

[0003] In the process of implementing the present invention, it is found that there are at least the following problems in the prior art: during the full backup process, if a node is offline, data loading will fail, and data backup needs to be continued by breakpoint resumption, which is complicated to operate and has low backup efficiency. Summary of the invention

[0004] In view of the above problems, the present invention provides a data incremental synchronization method, apparatus, device and storage medium.

[0005] According to a first aspect of the present invention, a method for incremental data synchronization is provided, comprising: in response to a data change in a target data table stored in a target database cluster, determining a cluster to be updated of the target database cluster based on a cluster topology graph, wherein the cluster topology graph is a directed acyclic graph, comprising a plurality of nodes and a plurality of directed edges, the plurality of nodes respectively corresponding to a plurality of database clusters in the database, each directed edge pointing from a parent node to a child node, the child node being the cluster to be updated of the parent node; determining a data structure to be updated of the data table to be updated stored in the cluster to be updated, a target data structure of the target data table, and an update range of the data table to be updated, wherein the data structure to be updated includes a data partition of the data table to be updated, the target data structure includes a data partition of the target data table, and the update range includes one or more data table columns to be synchronized during data synchronization; updating the data structure to be updated based on the target data structure and the data structure to be updated to obtain an updated data structure; determining an incremental synchronization statement corresponding to each data partition in the updated data structure based on the target data table, the data table to be updated, and the update range; and executing incremental synchronization statements corresponding to the data partitions on the plurality of data partitions of the data table to be updated, respectively, to achieve incremental data synchronization.

[0006] According to an embodiment of the present invention, the cluster to be updated includes a backup cluster and a reference cluster, wherein the backup cluster is used to back up the target data table, and the reference cluster is used to reference one or more data table columns in the target data table; in response to a data change in the target data table stored in the target database cluster, based on the cluster topology map, the cluster to be updated of the target database cluster is determined, including: determining a target node corresponding to the target database cluster in the cluster topology map; determining one or more rear-drive edges of the target node based on the cluster topology map; and determining the node in the cluster topology map with each rear-drive edge as a front-drive edge as the cluster to be updated.

[0007] According to an embodiment of the present invention, determining the data structure to be updated of the data table to be updated stored in the cluster to be updated, the target data structure of the target data table, and the update range of the data table to be updated includes: parsing the data table to be updated and the target data table to determine the data structure to be updated and the target data structure; when the cluster to be updated is a backup cluster, determining the update range to be all data table columns of the target data table; and when the cluster to be updated is a reference cluster, determining the update range to be one or more data table columns of the target data table referenced by the cluster to be updated.

[0008] According to an embodiment of the present invention, based on the target data structure and the data structure to be updated, the data structure to be updated is updated to obtain an updated data structure, including: based on the target data structure and the data structure to be updated, determining structural differences, wherein the structural differences include partitions to be inserted and redundant partitions, the partitions to be inserted include more partitions in the target data structure than the data structure to be updated, and the redundant partitions include more partitions in the data structure to be updated than the target data structure; based on the structural differences, generating partition update statements, wherein the partition update statements include statements for generating partitions to be inserted and statements for deleting redundant partitions; and executing the partition update statements on the cluster to be updated to update the data structure to be updated to obtain an updated data structure.

[0009] According to an embodiment of the present invention, based on a target data table, a data table to be updated and an update range, an incremental synchronization statement corresponding to each data partition in an update data structure is determined, including: based on the target data table and the data table to be updated, from multiple partitions of the target data table, determining one or more update partitions to be updated; based on the one or more update partitions and the update range, determining a data range to be updated in each update partition, wherein the data range includes data in a data table column corresponding to the update range in the update partition; and generating an incremental synchronization statement based on the data included in the data range to be updated.

[0010] According to an embodiment of the present invention, based on a target data table and a data table to be updated, one or more update partitions to be updated are determined from multiple partitions of the target data table, including: for each partition, when the partition is not a partition to be inserted, determining the data of the partition in the target data table and the data of the partition in the data table to be updated; when there is a difference between the data of the partition in the target data table and the data of the partition in the data table to be updated, determining the partition to be an update partition; and when the partition is a partition to be inserted, determining the partition to be an update partition.

[0011] According to an embodiment of the present invention, the incremental data synchronization method also includes: in the case where a new data table exists in the database, determining a new node corresponding to the new data table; based on the relationship between the new data table and multiple data tables already existing in the database, determining one or more new directed edges between the new node and multiple nodes; and updating the cluster topology diagram based on the new node and the one or more new directed edges.

[0012] A second aspect of the present invention provides a data incremental synchronization device, comprising:

[0013] A cluster determination module is used to determine the cluster to be updated of the target database cluster based on the cluster topology diagram in response to data changes in the target data table stored in the target database cluster, wherein the cluster topology diagram is a directed acyclic graph, including multiple nodes and multiple directed edges, the multiple nodes respectively correspond to the multiple database clusters in the database, each directed edge points from a parent node to a child node, and the child node is the cluster to be updated of the parent node;

[0014] A range determination module, used to determine the to-be-updated data structure of the to-be-updated data table stored in the to-be-updated cluster, the target data structure of the target data table, and the update range of the to-be-updated data table, wherein the to-be-updated data structure includes the data partitions of the to-be-updated data table, the target data structure includes the data partitions of the target data table, and the update range includes one or more data table columns that need to be synchronized during data synchronization;

[0015] A structure updating module, used for updating the data structure to be updated based on the target data structure and the data structure to be updated, to obtain an updated data structure;

[0016] A statement generation module, used to determine the incremental synchronization statement corresponding to each data partition in the update data structure based on the target data table, the data table to be updated and the update range;

[0017] The data synchronization module is used to execute the incremental synchronization statements corresponding to the data partitions on the multiple data partitions of the data table to be updated, so as to realize the incremental data synchronization.

[0018] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0019] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0020] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0021] According to an embodiment of the present invention, in response to a data change in a target data table in a target database, a cluster to be updated corresponding to the target database cluster is quickly determined based on a cluster topology diagram. The target data structure of the target data table, the data structure to be updated of the data table to be updated stored in the cluster to be updated, and the update range of the data table to be updated are determined respectively, and the data structure to be updated is updated so that the updated data structure has the same partition as the target data structure, so that data synchronization can be performed based on the partition later, thereby improving the efficiency of data synchronization. Based on the target data table, the data table to be updated, and the update range, the data table rows and data table columns that need to be synchronized are determined, so that the data to be synchronized in the data structure is updated, and an incremental synchronization statement is generated, and the incremental synchronization statement is executed on the data partition to achieve incremental data synchronization. Since the data table rows that need to be synchronized are determined by the relationship between the target data table and the data table to be updated, and the data table columns are determined by the update range, the data that needs to be synchronized are all necessary data for data synchronization, which reduces the synchronization process of redundant data, thereby reducing the amount of data processing, and can further improve the efficiency of data synchronization under the premise of ensuring the correctness of data synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0023] Figure 1 An application scenario diagram of a data incremental synchronization method, apparatus, device, and storage medium according to an embodiment of the present invention is shown.

[0024] Figure 2 A flow chart of a method for synchronizing incremental data according to an embodiment of the present invention is shown.

[0025] Figure 3 A schematic diagram of a cluster topology diagram according to an embodiment of the present invention is shown.

[0026] Figure 4 The reference relationship of data table columns between the target data table and the data table of the reference cluster according to an embodiment of the present invention is shown.

[0027] Figure 5 A structural block diagram of a data incremental synchronization device according to an embodiment of the present invention is shown.

[0028] Figure 6 A block diagram of an electronic device suitable for implementing a data incremental synchronization method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, invention and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0035] An embodiment of the present invention provides a data incremental synchronization method, comprising: in response to a data change in a target data table stored in a target database cluster, determining a cluster to be updated of the target database cluster based on a cluster topology diagram, wherein the cluster topology diagram is a directed acyclic graph, including multiple nodes and multiple directed edges, the multiple nodes respectively correspond to multiple database clusters in the database, each directed edge points from a parent node to a child node, and the child node is the cluster to be updated of the parent node; determining a data structure to be updated of the data table to be updated stored in the cluster to be updated, a target data structure of the target data table, and an update range of the data table to be updated, wherein the data structure to be updated includes a data partition of the data table to be updated, the target data structure includes a data partition of the target data table, and the update range includes one or more data table columns that need to be synchronized during data synchronization; based on the target data structure and the data structure to be updated, updating the data structure to be updated to obtain an updated data structure; based on the target data table, the data table to be updated, and the update range, determining an incremental synchronization statement corresponding to each data partition in the updated data structure; and on multiple data partitions of the data table to be updated, respectively executing incremental synchronization statements corresponding to the data partitions to achieve incremental data synchronization.

[0036] Figure 1 An application scenario diagram of a data incremental synchronization method, apparatus, device, and storage medium according to an embodiment of the present invention is shown.

[0037] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0038] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0040] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0041] It should be noted that the data increment synchronization method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the data increment synchronization device provided in the embodiment of the present invention can generally be set in the server 105. The data increment synchronization method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the data increment synchronization device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0042] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0043] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The data incremental synchronization method according to an embodiment of the present invention is described in detail.

[0044] Figure 2 A flow chart of a method for synchronizing incremental data according to an embodiment of the present invention is shown.

[0045] like Figure 2 As shown, the data incremental synchronization method of this embodiment includes operations S210 to S250.

[0046] In operation S210 , in response to a data change in a target data table stored in a target database cluster, a cluster to be updated of the target database cluster is determined based on a cluster topology diagram.

[0047] According to an embodiment of the present invention, a database system includes multiple database clusters, each of which stores a data table. When data changes in a data table in the database system, the data table in which the data change occurs can be determined based on the log of the database system and is determined as a target data table, and the database cluster storing the target data table is determined as the target database cluster.

[0048] According to an embodiment of the present invention, the cluster topology graph is a directed acyclic graph, including multiple nodes and multiple directed edges, the multiple nodes respectively correspond to multiple database clusters in the database, each directed edge points from a parent node to a child node, and the child node is the cluster to be updated of the parent node.

[0049] According to an embodiment of the present invention, based on the cluster topology diagram, a node corresponding to the target database cluster is determined and one or more child nodes of the node are determined, and the one or more child nodes are determined as the cluster to be updated.

[0050] In operation S220 , the to-be-updated data structure of the to-be-updated data table stored in the to-be-updated cluster, the target data structure of the target data table, and the update range of the to-be-updated data table are determined.

[0051] According to an embodiment of the present invention, the data structure to be updated includes data partitions of the data table to be updated, the target data structure includes data partitions of the target data table, and the update range includes one or more data table columns that need to be synchronized during data synchronization. Data partitions are multiple partitions obtained by dividing the data in the data table according to the range of attribute values ​​of a certain column.

[0052] According to an embodiment of the present invention, the structure acquisition instruction can be used to acquire the data structure to be updated and the target data structure based on the data table to be updated and the target data table, respectively. According to the relationship between the cluster to be updated and the target database cluster, the data table columns to be updated in the data table to be updated can be determined, thereby determining the update scope.

[0053] In operation S230 , based on the target data structure and the data structure to be updated, the data structure to be updated is updated to obtain an updated data structure.

[0054] According to an embodiment of the present invention, based on the target data structure and the data structure to be updated, the difference between the target data structure and the data structure to be updated is determined, and the data structure to be updated is updated so that the data partition included in the data structure to be updated is the same as that in the target data structure, thereby obtaining an updated data structure.

[0055] In operation S240 , an incremental synchronization statement corresponding to each data partition in the update data structure is determined based on the target data table, the to-be-updated data table, and the update range.

[0056] According to an embodiment of the present invention, based on the target data table, the data table to be updated and the update range, the data that needs to be synchronized in the update data structure is determined, and based on the attribute values ​​of the data in the target data table, an incremental synchronization statement for inserting the above data into the update data structure is generated.

[0057] In operation S250, incremental synchronization statements corresponding to the data partitions are respectively executed on the multiple data partitions of the data table to be updated to implement incremental data synchronization.

[0058] According to an embodiment of the present invention, after the incremental synchronization statement is executed, the data corresponding to the incremental synchronization statement will be inserted or updated to the corresponding data partition, thereby achieving data synchronization with the target data table.

[0059] According to an embodiment of the present invention, in response to a data change in a target data table in a target database, a cluster to be updated corresponding to the target database cluster is quickly determined based on a cluster topology diagram. The target data structure of the target data table, the data structure to be updated of the data table to be updated stored in the cluster to be updated, and the update range of the data table to be updated are determined respectively, and the data structure to be updated is updated so that the updated data structure has the same partition as the target data structure, so that data synchronization can be performed based on the partition later, thereby improving the efficiency of data synchronization. Based on the target data table, the data table to be updated, and the update range, the data table rows and data table columns that need to be synchronized are determined, so that the data to be synchronized in the data structure is updated, and an incremental synchronization statement is generated, and the incremental synchronization statement is executed on the data partition to achieve incremental data synchronization. Since the data table rows that need to be synchronized are determined by the relationship between the target data table and the data table to be updated, and the data table columns are determined by the update range, the data that needs to be synchronized are all necessary data for data synchronization, which reduces the synchronization process of redundant data, thereby reducing the amount of data processing, and can further improve the efficiency of data synchronization under the premise of ensuring the correctness of data synchronization.

[0060] According to an embodiment of the present invention, the cluster to be updated includes a backup cluster and a reference cluster, wherein the backup cluster is used to back up the target data table, and the reference cluster is used to reference one or more data table columns in the target data table; in response to a data change in the target data table stored in the target database cluster, based on the cluster topology map, the cluster to be updated of the target database cluster is determined, including: determining a target node corresponding to the target database cluster in the cluster topology map; determining one or more rear-drive edges of the target node based on the cluster topology map; and determining the node in the cluster topology map with each rear-drive edge as a front-drive edge as the cluster to be updated.

[0061] According to an embodiment of the present invention, since the backup cluster is used to back up the target data table, when data changes such as data insertion and data modification occur in the target data table, the backup data table stored in the backup cluster needs to perform incremental synchronization on all data changes.

[0062] According to an embodiment of the present invention, since the reference cluster is used to reference one or more data table columns in the target data table, when data is inserted into the target data table, the data tables in the reference cluster do not need to synchronize data. In the target data table, when data is modified in the data table columns referenced by the data tables of the referenced cluster, the data tables in the reference cluster need to synchronize data corresponding to the referenced data table columns.

[0063] For example, the foreign key of a data table in the referenced cluster is the primary key of a target data table, and the data table in the referenced cluster also references data table column X in the target data table. If data in data table column X is modified, the data table in the referenced cluster needs to modify the attribute value in the data table column based on the correspondence between the attribute value of data table column X and the primary key of the target data table.

[0064] According to an embodiment of the present invention, a target node corresponding to a target database cluster is determined in a cluster topology map, one or more rear-drive edges of the target node are determined in the cluster topology map, and for one or more rear-drive edges, a node with each rear-drive edge as a front-drive edge is determined in the cluster topology map, and the database cluster corresponding to the node is determined as a cluster to be updated, wherein the rear-drive edge of a node is an edge starting from the node and pointing to other nodes, and the front-drive edge of a node is an edge pointing to the node. That is, the rear-drive node of the target node is determined from the cluster topology map, and the database cluster corresponding to the rear-drive node is determined as the cluster to be updated.

[0065] According to an embodiment of the present invention, a target node corresponding to a target database cluster is determined in a cluster topology map, and a successor node of the target node is determined as a cluster to be updated based on the cluster topology map. This enables the cluster to be updated to be determined quickly and accurately, and subsequent operations to be performed, thereby improving the efficiency and accuracy of data synchronization.

[0066] Figure 3 A schematic diagram of a cluster topology diagram according to an embodiment of the present invention is shown.

[0067] like Figure 3 As shown, the cluster topology diagram includes six nodes, among which Cluster 1 and Cluster 2' are the rear-drive nodes of Cluster 2. Therefore, Cluster 1 and Cluster 2' are the clusters to be updated of Cluster 2. Similarly, Cluster 1', Cluster 3 and Cluster 4 are the clusters to be updated of Cluster 1, among which Cluster 2' is the backup cluster of Cluster 2, and Cluster 1' is the backup cluster of Cluster 1.

[0068] When the data in the data table stored in cluster 2 changes, according to the cluster topology diagram, cluster 2' and cluster 1 can be determined as clusters to be updated, so data synchronization is performed on cluster 2' and cluster 1. After cluster 1 completes data synchronization, cluster 1 is used as the target database cluster. According to the cluster topology diagram, cluster 1', cluster 3 and cluster 4 can be determined as clusters to be updated, and data synchronization is performed on cluster 1', cluster 3 and cluster 4. Therefore, data synchronization can be achieved in parallel for multiple clusters to be updated of the same target database cluster, thereby improving the efficiency and accuracy of data synchronization.

[0069] According to an embodiment of the present invention, determining the data structure to be updated of the data table to be updated stored in the cluster to be updated, the target data structure of the target data table, and the update range of the data table to be updated includes: parsing the data table to be updated and the target data table to determine the data structure to be updated and the target data structure; when the cluster to be updated is a backup cluster, determining the update range to be all data table columns of the target data table; and when the cluster to be updated is a reference cluster, determining the update range to be one or more data table columns of the target data table referenced by the cluster to be updated.

[0070] According to an embodiment of the present invention, the data table to be updated and the target data table are parsed, and data partitions included in the data table to be updated and the target data table are determined respectively to obtain the data structure to be updated and the target data structure.

[0071] According to an embodiment of the present invention, when the cluster to be updated is a backup cluster, it is necessary to synchronize the data in the entire data table where data changes occur, so the update range is all data table columns of the target data table.

[0072] According to an embodiment of the present invention, when the cluster to be updated is a reference cluster, it is necessary to update the data table columns referenced by the data table in the reference cluster, and one or more referenced data table columns are determined from the target data table as the update range.

[0073] According to an embodiment of the present invention, the category of the cluster to be updated is determined. When the cluster to be updated is a backup cluster, the update scope is determined to be all the data table columns of the target data table, thereby realizing incremental synchronization of all data in the target data table, ensuring that the data stored in the backup cluster is completely consistent with the data stored in the target database cluster, and ensuring data security. When the cluster to be updated is a reference cluster, based on the reference of the data table in the reference cluster to the target data table, the referenced data table columns are determined as the update scope, and the data table columns involved in data synchronization are minimized while ensuring that the synchronization is performed correctly, thereby reducing the amount of data processing, improving efficiency, and reducing load.

[0074] Figure 4 The reference relationship of data table columns between the target data table and the data table of the reference cluster according to an embodiment of the present invention is shown.

[0075] like Figure 4 As shown, Table 1 is a part of the student information table, Table 2 is a part of the major information table, the primary key of Table 2 is "major code", the primary key of Table 1 is "student number", the foreign key is "major code", and Table 1 references the data table column "major director" of Table 2, that is, the database cluster that saves Table 1 is the reference cluster of the database cluster that saves Table 2, and the reference range includes the two data table columns "major code" and "major director" of Table 2.

[0076] When the data in Table 2 changes, for example, the director of major B changes from Sun Qi to Zhou Ba, according to the reference relationship between the data tables, it can be determined that the update range includes the two data table columns "major code" and "major director". During the data synchronization process, the data in the second and third rows of Table 1 will be updated according to the updated information in Table 2.

[0077] According to an embodiment of the present invention, based on the target data structure and the data structure to be updated, the data structure to be updated is updated to obtain an updated data structure, including: based on the target data structure and the data structure to be updated, determining structural differences, wherein the structural differences include partitions to be inserted and redundant partitions, the partitions to be inserted include more partitions in the target data structure than the data structure to be updated, and the redundant partitions include more partitions in the data structure to be updated than the target data structure; based on the structural differences, generating partition update statements, wherein the partition update statements include statements for generating partitions to be inserted and statements for deleting redundant partitions; and executing the partition update statements on the cluster to be updated to update the data structure to be updated to obtain an updated data structure.

[0078] According to an embodiment of the present invention, multiple data partitions included in the target data structure and the data structure to be updated are compared, and the data partition that exists in the target data structure but not in the data structure to be updated is determined as the partition to be inserted, and the data partition that exists in the data structure to be updated but not in the target data structure is determined as a redundant partition, and the structural differences are determined based on the partition to be inserted and the redundant partition.

[0079] According to an embodiment of the present invention, based on the structural difference, a statement for inserting the partition to be inserted into the cluster to be updated and a statement for deleting the redundant partition in the cluster to be updated are generated to obtain a partition update statement. The partition update statement is executed on the cluster to be updated, the partition to be inserted is inserted into the cluster to be updated, and the redundant partition in the cluster to be updated and the data included in the redundant partition are deleted to obtain an updated update data structure, wherein the partition included in the update data structure is the same as the partition included in the target data structure.

[0080] According to an embodiment of the present invention, based on the target data structure and the data structure to be updated, the structural difference between the two is determined, and a statement for inserting the partition to be inserted and a statement for deleting the redundant partition are generated. The above statements are executed on the cluster to be updated to obtain an updated data structure, which can ensure that the updated data structure has the same partition as the target data structure, thereby improving the accuracy of subsequent intra-partition data comparison and further improving the efficiency of data synchronization.

[0081] According to an embodiment of the present invention, based on a target data table, a data table to be updated and an update range, an incremental synchronization statement corresponding to each data partition in an update data structure is determined, including: based on the target data table and the data table to be updated, from multiple partitions of the target data table, determining one or more update partitions to be updated; based on the one or more update partitions and the update range, determining a data range to be updated in each update partition, wherein the data range includes data in a data table column corresponding to the update range in the update partition; and generating an incremental synchronization statement based on the data included in the data range to be updated.

[0082] According to an embodiment of the present invention, for each partition in the update data structure, it is determined whether the data in the partition is the same in the target data table and the data table to be updated. If the data is different, it is determined that the partition has been updated and the partition is determined as an updated partition. If the data is the same, it is determined that the partition has not been updated.

[0083] According to an embodiment of the present invention, after determining the update partition, based on the data of the partition in the update data structure and the data of the partition in the target data table, the data table rows that need to be synchronized are determined using the data comparison method of incremental synchronization, and based on the update range, the data table columns that need to be synchronized in the update partition are determined, thereby determining the data range that needs to be updated based on the overlapping part of the data table rows and data table columns, and determining the attribute values ​​of the data in the data range that needs to be updated based on the target data table, and generating incremental synchronization statements based on the data range and the attribute values.

[0084] According to an embodiment of the present invention, based on the data conditions of the update partition in the target data table and the update data structure, it is determined that there is an update and the data table row that needs to be synchronized is determined, based on the update range, the data table column that needs to be synchronized is determined, and based on the overlapping part of the data table row and the data table column, the data range that needs to be updated is determined. Data synchronization can be accurately limited to the minimum range, thereby reducing the amount of data processing and achieving fast incremental synchronization. Based on the data range, the attribute value of the data in the range is determined from the target data table, and an incremental synchronization statement is generated to ensure the accuracy of incremental synchronization.

[0085] According to an embodiment of the present invention, based on a target data table and a data table to be updated, one or more update partitions to be updated are determined from multiple partitions of the target data table, including: for each partition, when the partition is not a partition to be inserted, determining the data of the partition in the target data table and the data of the partition in the data table to be updated; when there is a difference between the data of the partition in the target data table and the data of the partition in the data table to be updated, determining the partition to be an update partition; and when the partition is a partition to be inserted, determining the partition to be an update partition.

[0086] According to an embodiment of the present invention, since data is stored in segments in different partitions, in the event of data changes, it is possible to first determine whether the data in each partition has changed, wherein the data in the partitions where the data has not changed does not need to be synchronized, and only the data in the partitions where the data has changed needs to be synchronized.

[0087] For each partition, determine whether the partition is a partition to be inserted. If the partition is not a partition to be inserted, compare the data of the partition in the target data table and the partition corresponding to the partition in the data table to be updated. If there is no difference in the data of the two partitions, it indicates that the data in the partition has not changed and no data synchronization is required. If there is a difference in the data of the two partitions, it indicates that data synchronization is required for the partition and the partition is determined as an update partition.

[0088] According to an embodiment of the present invention, when the partition is a partition to be inserted, it is proved that the partition is not included in the data table to be updated. The partition is newly added during this data change, so the data in the partition is all new data. The partition needs to perform data synchronization, so the partition is determined as an update partition.

[0089] According to an embodiment of the present invention, for each partition, it is determined whether there is a difference between the data stored in the target data table and the data table to be updated for the partition, and the partition with the difference is determined as the update partition for data synchronization. The partition without the difference does not need to undergo a subsequent synchronization process, which can reduce the partitions involved in data synchronization, thereby improving the processing efficiency of incremental data synchronization and reducing the data processing load.

[0090] According to an embodiment of the present invention, the incremental data synchronization method also includes: in the case where a new data table exists in the database, determining a new node corresponding to the new data table; based on the relationship between the new data table and multiple data tables already existing in the database, determining one or more new directed edges between the new node and multiple nodes; and updating the cluster topology diagram based on the new node and the one or more new directed edges.

[0091] According to an embodiment of the present invention, when a new data table exists in the database, the cluster topology map can be updated to ensure that data synchronization can be correctly performed using the cluster topology map. A new node corresponding to the new data table is added to the cluster topology map.

[0092] According to an embodiment of the present invention, based on the dependency relationship between the newly created data table and other data tables in the database, one or more newly created directed edges between the newly created node and the other multiple nodes are determined, wherein, for a referenced data table referenced by the newly created data table, a directed edge points from a node corresponding to the referenced data table to the newly created node, and for other data tables that reference the newly created data table, a directed edge points from the newly created node to nodes corresponding to other data tables that reference the newly created data table.

[0093] According to an embodiment of the present invention, based on the newly created data table, a data table for backing up the data in the newly created data table can be created in the database system, a node corresponding to the data table for backing up the data in the newly created data table can be created in the cluster topology map, and the node can be used as the follow-up node of the newly created node to update the cluster topology map.

[0094] According to an embodiment of the present invention, when a new data table exists in a database, a new node corresponding to the new data table and a node corresponding to a data table used to back up the new data table are added to the cluster topology map, and based on the relationship between the new data table and other data tables in the database, the directed edges between the new node and other nodes are determined, and the cluster topology map is updated. It is ensured that the cluster to be updated can be correctly obtained when the cluster topology map is used for data synchronization, thereby ensuring that data synchronization can be performed correctly.

[0095] Based on the above data increment synchronization method, the present invention also provides a data increment synchronization device. Figure 5 The device is described in detail.

[0096] Figure 5 A structural block diagram of a data incremental synchronization device according to an embodiment of the present invention is shown.

[0097] like Figure 5 As shown, the data incremental synchronization device 500 of this embodiment includes a cluster determination module 510 , a range determination module 520 , a structure update module 530 , a statement generation module 540 and a data synchronization module 550 .

[0098] The cluster determination module 510 is used to determine the cluster to be updated of the target database cluster based on the cluster topology diagram in response to the data change of the target data table stored in the target database cluster, wherein the cluster topology diagram is a directed acyclic graph, including multiple nodes and multiple directed edges, the multiple nodes respectively correspond to the multiple database clusters in the database, each directed edge points from the parent node to the child node, and the child node is the cluster to be updated of the parent node. In one embodiment, the cluster determination module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0099] The range determination module 520 is used to determine the data structure to be updated of the data table to be updated stored in the cluster to be updated, the target data structure of the target data table, and the update range of the data table to be updated, wherein the data structure to be updated includes the data partition of the data table to be updated, the target data structure includes the data partition of the target data table, and the update range includes one or more data table columns that need to be synchronized during data synchronization. In one embodiment, the range determination module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0100] The structure updating module 530 is used to update the data structure to be updated based on the target data structure and the data structure to be updated to obtain an updated data structure. In one embodiment, the structure updating module 530 can be used to perform the operation S230 described above, which will not be described in detail here.

[0101] The statement generation module 540 is used to determine the incremental synchronization statement corresponding to each data partition in the update data structure based on the target data table, the data table to be updated and the update range. In one embodiment, the statement generation module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0102] The data synchronization module 550 is used to execute the incremental synchronization statements corresponding to the data partitions on the multiple data partitions of the data table to be updated, so as to realize the incremental data synchronization. In one embodiment, the data synchronization module 550 can be used to execute the operation S250 described above, which will not be described in detail here.

[0103] According to an embodiment of the present invention, the cluster determination module 510 includes a node determination submodule, a successor edge determination submodule and a cluster determination submodule.

[0104] The node determination submodule is used to determine the target node corresponding to the target database cluster in the cluster topology diagram.

[0105] The follower edge determination submodule is used to determine one or more follower edges of the target node based on the cluster topology graph.

[0106] The cluster determination submodule is used to determine the nodes in the cluster topology graph that use each successor edge as a predecessor edge as the cluster to be updated.

[0107] According to an embodiment of the present invention, the range determination module 520 includes a structure determination submodule, a first range determination submodule and a second range determination submodule.

[0108] The structure determination submodule is used to parse the data table to be updated and the target data table to determine the data structure to be updated and the target data structure.

[0109] The first range determination submodule is used to determine the update range as all data table columns of the target data table when the cluster to be updated is a backup cluster.

[0110] The second scope determination submodule is used to determine, when the cluster to be updated is a reference cluster, that the update scope is one or more data table columns of the reference target data table of the cluster to be updated.

[0111] According to an embodiment of the present invention, the structure updating module 530 includes a difference determining submodule, a partition statement generating submodule and a structure updating submodule.

[0112] The difference determination submodule is used to determine the structural difference based on the target data structure and the data structure to be updated, wherein the structural difference includes the partition to be inserted and the redundant partition, the partition to be inserted includes the partition in the target data structure that is more than the partition in the data structure to be updated, and the redundant partition includes the partition in the data structure to be updated that is more than the partition in the target data structure.

[0113] The partition statement generation submodule is used to generate partition update statements based on structural differences, wherein the partition update statements include statements for generating partitions to be inserted and statements for deleting redundant partitions.

[0114] The structure update submodule is used to execute partition update statements on the cluster to be updated, update the data structure to be updated, and obtain an updated data structure.

[0115] According to an embodiment of the present invention, the statement generation module 540 includes a partition determination submodule, a range determination submodule and a statement generation submodule.

[0116] The partition determination submodule is used to determine one or more update partitions to be updated from multiple partitions of the target data table based on the target data table and the data table to be updated.

[0117] The range determination submodule is used to determine the data range that needs to be updated in each update partition based on one or more update partitions and update ranges, wherein the data range includes data in the data table column corresponding to the update range in the update partition.

[0118] The statement generation submodule is used to generate incremental synchronization statements based on the data included in the data range that needs to be updated.

[0119] According to an embodiment of the present invention, the partition determination submodule includes a data determination unit, a first partition determination unit and a second partition determination unit.

[0120] The data determination unit is used to determine, for each partition, the data of the partition in the target data table and the data of the partition in the data table to be updated when the partition is not the partition to be inserted.

[0121] The first partition determination unit is configured to determine a partition as an update partition when there is a difference between the data of the partition in the target data table and the data of the partition in the to-be-updated data table.

[0122] The second partition determination unit is used to determine the partition as an update partition when the partition is a partition to be inserted.

[0123] According to an embodiment of the present invention, the data incremental synchronization device 500 further includes a node determination module, a directed edge determination module and a topology update module.

[0124] The node determination module is used to determine a new node corresponding to a new data table when a new data table exists in the database.

[0125] The directed edge determination module is used to determine one or more newly created directed edges between the newly created node and multiple nodes based on the relationship between the newly created data table and multiple data tables already existing in the database.

[0126] The topology update module is used to update the cluster topology graph based on the newly created nodes and one or more newly created directed edges.

[0127] According to an embodiment of the present invention, any multiple modules among the cluster determination module 510, the scope determination module 520, the structure update module 530, the statement generation module 540 and the data synchronization module 550 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the cluster determination module 510, the scope determination module 520, the structure update module 530, the statement generation module 540 and the data synchronization module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the cluster determination module 510 , the scope determination module 520 , the structure update module 530 , the statement generation module 540 , and the data synchronization module 550 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0128] Figure 6 A block diagram of an electronic device suitable for implementing a data incremental synchronization method according to an embodiment of the present invention is shown.

[0129] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0130] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0131] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0132] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0133] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0134] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0135] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0136] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0137] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0138] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0139] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0141] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data incremental synchronization method, characterized in that: The method comprises: In response to a data change in a target data table stored in a target database cluster, a cluster to be updated of the target database cluster is determined based on a cluster topology graph, wherein the cluster topology graph is a directed acyclic graph, including a plurality of nodes and a plurality of directed edges, the plurality of nodes respectively correspond to a plurality of database clusters in a database, each of the directed edges points from a parent node to a child node, the child node is the cluster to be updated of the parent node, the cluster to be updated includes a backup cluster and a reference cluster, the backup cluster is used to back up the target data table, and the reference cluster is used to reference one or more data table columns in the target data table; Determine the data structure to be updated of the data table to be updated stored in the cluster to be updated, the target data structure of the target data table, and the update range of the data table to be updated, wherein the data structure to be updated includes the data partitions of the data table to be updated, the target data structure includes the data partitions of the target data table, and the update range includes one or more data table columns that need to be synchronized during data synchronization; Based on the target data structure and the data structure to be updated, the data structure to be updated is updated to obtain an updated data structure; Determining, based on the target data table, the to-be-updated data table and the update range, an incremental synchronization statement corresponding to each of the data partitions in the update data structure; and On the multiple data partitions of the data table to be updated, respectively execute the incremental synchronization statements corresponding to the data partitions to achieve incremental data synchronization; The step of determining the to-be-updated data structure of the to-be-updated data table stored in the to-be-updated cluster, the target data structure of the target data table, and the update range of the to-be-updated data table includes: Parsing the to-be-updated data table and the target data table to determine the to-be-updated data structure and the target data structure; In the case where the cluster to be updated is the backup cluster, determining the update range to be all data table columns of the target data table; and In the case where the cluster to be updated is the reference cluster, determining the update range to be one or more data table columns of the target data table referenced by the cluster to be updated; The updating of the data structure to be updated based on the target data structure and the data structure to be updated to obtain an updated data structure includes: Determine structural differences based on the target data structure and the data structure to be updated, wherein the structural differences include partitions to be inserted and redundant partitions, the partitions to be inserted include more partitions in the target data structure than the data structure to be updated, and the redundant partitions include more partitions in the data structure to be updated than the target data structure; Based on the structural difference, generating a partition update statement, wherein the partition update statement includes a statement for generating the partition to be inserted and a statement for deleting the redundant partition; and Executing the partition update statement on the cluster to be updated, updating the data structure to be updated, and obtaining an updated data structure; The step of determining, based on the target data table, the to-be-updated data table, and the update range, an incremental synchronization statement corresponding to each of the data partitions in the update data structure comprises: Based on the target data table and the to-be-updated data table, determining one or more update partitions to be updated from multiple partitions of the target data table; Determining a data range that needs to be updated in each update partition based on the one or more update partitions and the update range, wherein the data range includes data in a data table column corresponding to the update range in the update partition; and The incremental synchronization statement is generated based on the data included in the data range that needs to be updated.

2. The method according to claim 1, characterized in that Said In response to a data change in a target data table stored in a target database cluster, determining a cluster to be updated of the target database cluster based on a cluster topology diagram includes: Determine a target node corresponding to the target database cluster in the cluster topology diagram; Based on the cluster topology graph, determining one or more downstream edges of the target node; and A node in the cluster topology graph that uses each of the successor edges as a predecessor edge is determined as the cluster to be updated.

3. The method according to claim 1, characterized in that The determining, based on the target data table and the to-be-updated data table, one or more update partitions to be updated from the multiple partitions of the target data table comprises: For each of the partitions, In the case where the partition is not the partition to be inserted, determining the data of the partition in the target data table and the data of the partition in the data table to be updated; In the case where there is a difference between the data of the partition in the target data table and the data of the partition in the to-be-updated data table, determining the partition as the update partition; and In the case that the partition is a partition to be inserted, the partition is determined to be the update partition.

4. The method according to claim 1, characterized in that: The method further comprises: If a new data table exists in the database, determining a new node corresponding to the new data table; Based on the relationship between the newly created data table and the multiple data tables already existing in the database, determining one or more newly created directed edges between the newly created node and the multiple nodes; and The cluster topology graph is updated based on the newly created node and the one or more newly created directed edges.

5. A data incremental synchronization device, characterized in that: The device comprises: A cluster determination module, configured to determine, in response to a data change in a target data table stored in a target database cluster, a cluster to be updated of the target database cluster based on a cluster topology diagram, wherein the cluster topology diagram is a directed acyclic graph, including a plurality of nodes and a plurality of directed edges, the plurality of nodes respectively corresponding to a plurality of database clusters in a database, each of the directed edges pointing from a parent node to a child node, the child node being the cluster to be updated of the parent node, the cluster to be updated including a backup cluster and a reference cluster, the backup cluster being used to back up the target data table, the reference cluster being used to reference one or more data table columns in the target data table; A range determination module, used to determine the to-be-updated data structure of the to-be-updated data table stored in the to-be-updated cluster, the target data structure of the target data table, and the update range of the to-be-updated data table, wherein the to-be-updated data structure includes the data partitions of the to-be-updated data table, the target data structure includes the data partitions of the target data table, and the update range includes one or more data table columns that need to be synchronized during data synchronization; A structure updating module, used for updating the data structure to be updated based on the target data structure and the data structure to be updated to obtain an updated data structure; a statement generation module, configured to determine, based on the target data table, the to-be-updated data table and the update range, an incremental synchronization statement corresponding to each of the data partitions in the update data structure; and A data synchronization module, used to respectively execute incremental synchronization statements corresponding to the data partitions on the multiple data partitions of the data table to be updated, so as to realize incremental data synchronization; Wherein, the range determination module includes: A structure determination submodule, used for parsing the to-be-updated data table and the target data table to determine the to-be-updated data structure and the target data structure; A first range determination submodule, configured to determine, when the cluster to be updated is the backup cluster, that the update range is all data table columns of the target data table; and A second range determination submodule, configured to determine, when the cluster to be updated is the reference cluster, that the update range is one or more data table columns of the target data table referenced by the cluster to be updated; The structure updating module comprises: a difference determination submodule, configured to determine a structural difference based on the target data structure and the data structure to be updated, wherein the structural difference includes a partition to be inserted and a redundant partition, the partition to be inserted includes more partitions in the target data structure than the data structure to be updated, and the redundant partition includes more partitions in the data structure to be updated than the target data structure; A partition statement generating submodule, used to generate a partition update statement based on the structural difference, wherein the partition update statement includes a statement for generating the partition to be inserted and a statement for deleting the redundant partition; and A structure update submodule, used for executing the partition update statement on the cluster to be updated, updating the data structure to be updated, and obtaining an updated data structure; The statement generation module comprises: A partition determination submodule, configured to determine, based on the target data table and the to-be-updated data table, one or more update partitions to be updated from the multiple partitions of the target data table; a range determination submodule, configured to determine a data range to be updated in each update partition based on the one or more update partitions and the update range, wherein the data range includes data in a data table column corresponding to the update range in the update partition; and The statement generation submodule is used to generate the incremental synchronization statement based on the data included in the data range that needs to be updated.

6. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images