Data synchronization method and device of database system, medium and product

Abstract

The invention provides a data synchronization method and device of a database system, a medium and a product. The data synchronization method of the database system comprises the following steps: reading a source end data block from a data file of a source end database according to a preset size; calculating a first source end verification value of the source end data block according to a first preset algorithm; checking whether the first source end verification value has mapping in the hash table, if not, judging that the source end data block is incremental data needing to be synchronized to the target end database, and sending the source end data block to the target end database; wherein the hash table is constructed according to data block information of the target end database, and the data block information comprises a unique number of each data block divided by the target end database according to a preset size and a first target end check value calculated according to a first preset algorithm. In the synchronization process, only the source end data block serving as the incremental data needs to be synchronized to the target end database, so that the amount of data needing to be transmitted is reduced, and the synchronization efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a data synchronization method, device, medium and product of a database system. Background Art

[0002] During the operation of a database system, data synchronization between two databases is a common work requirement. For example, when a database in a database cluster needs to be redone due to a failure, the data of other surviving databases in the database cluster will be synchronized to the failed database to achieve recovery of the failed database. Currently, when rebuilding a failed database, the data of the surviving database is usually copied to the failed database in full. When the database data volume is large and the cluster network bandwidth is limited, the database failure recovery time is greatly extended. Summary of the invention

[0003] An object of the present invention is to provide a data synchronization method, device, medium and product for a database system that can improve synchronization efficiency.

[0004] In particular, the present invention provides a data synchronization method for a database system, comprising:

[0005] Read source data blocks from data files in the source database according to a preset size;

[0006] Calculate a first source check value of the source data block according to a first preset algorithm;

[0007] Check whether the first source-end check value is mapped in the hash table; if not, determine that the source-end data block is incremental data that needs to be synchronized to the target-end database, and send the source-end data block to the target-end database;

[0008] The hash table is constructed according to data block information of the target database, and the data block information includes a unique number of each data block divided by the target database according to the preset size and a first target check value calculated according to the first preset algorithm.

[0009] Optionally, after the step of checking whether the first source-end verification value is mapped in the hash table, the step includes:

[0010] If so, searching for a corresponding first target-end check value in an information set according to the mapping value and comparing the first target-end check value with the first source-end check value, the information set being used to record data block information from the target-end database;

[0011] Determine whether the first source-side verification value and the first target-side verification value are the same. If so, determine that the source-side data block is non-incremental data that does not need to be synchronized to the target-side database. If not, determine that the source-side data block is incremental data that needs to be synchronized to the target-side database.

[0012] Optionally, the data block information further includes a second target end check value calculated according to a second preset algorithm;

[0013] If the first source-end check value is the same as the first target-end check value, the data synchronization method of the database system further includes:

[0014] Calculating a second source check value of the source data block according to the second preset algorithm;

[0015] Searching for a corresponding second target-end verification value in the information set according to the mapping value and comparing the second target-end verification value with the second source-end verification value;

[0016] Determine whether the second source-side verification value and the second target-side verification value are the same. If so, determine that the source-side data block is non-incremental data that does not need to be synchronized to the target-side database. If not, determine that the source-side data block is incremental data that needs to be synchronized to the target-side database.

[0017] Optionally, the data synchronization method of the database system includes:

[0018] Dividing the data files of the target database into data blocks according to the preset size and marking them with unique numbers in sequence;

[0019] Read and calculate the first target-end check value and the second target-end check value of each of the data blocks in sequence;

[0020] The data block information consisting of the unique number, the first target-end verification value and the second target-end verification value is sent to the source-end database.

[0021] Optionally, the data synchronization method of the database system includes:

[0022] Receiving data block information from the target database;

[0023] Constructing the information set according to the data block information;

[0024] The hash table is constructed according to the data block information, wherein the first target-end check value is converted into a key of the hash table.

[0025] Optionally, if the first source-end check value is different from the first target-end check value, the data synchronization method of the database system further includes:

[0026] Check whether there are conflicting mapping values ​​in the mapping position of the hash table. If so, search the corresponding first target-end verification value in the information set according to the conflicting mapping value and compare the first target-end verification value with the first source-end verification value, and execute the step of determining whether the first source-end verification value and the first target-end verification value are the same. If not, determine that the source-end data block is incremental data that needs to be synchronized to the target-end database.

[0027] Optionally, the data synchronization method of the database system includes:

[0028] Creating a temporary file in the target database;

[0029] Receiving incremental data blocks from the source database and storing them in the temporary file;

[0030] Receiving a unique number corresponding to a non-incremental data block from the source database and copying the data block corresponding to the target database to the temporary file;

[0031] It is detected that all information processing from the source database is completed, the original data file of the target database is deleted and the temporary file is renamed to the name of the original data file.

[0032] According to another aspect of the present invention, there is also provided a computer device, comprising a memory, a processor, and a computer executable program stored in the memory and running on the processor, and the processor implements the data synchronization method of the database system according to any one of the above items when executing the computer executable program.

[0033] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer-executable program is stored. When the computer-executable program is executed by a processor, the method for synchronizing data in a database system according to any one of the above items is implemented.

[0034] According to another aspect of the present invention, a computer program product is provided, including a computer executable program, and when the computer executable program is executed by a processor, the method for synchronizing data in a database system according to any one of the above items is implemented.

[0035] The data synchronization method, device, medium and product of the database system of the present invention read the source data block from the data file of the source database according to the preset size, calculate the first source check value of the source data block according to the first preset algorithm, check whether the first source check value is mapped in the hash table, if not, determine that the source data block is the incremental data that needs to be synchronized to the target database, and send the source data block to the target database. The hash table is constructed according to the data block information of the target database, and the data block information includes the unique number of each data block divided by the target database according to the preset size and the first target check value calculated according to the first preset algorithm. In other words, the source data block that the target database does not have can be compared according to the mapping of the first source check value of the source data block and the first target check value of the target data block in the hash table, that is, the incremental data of the source database relative to the target database can be compared. During synchronization, only the source data block as incremental data needs to be synchronized to the target database, which helps to reduce the amount of data that needs to be transmitted, saves transmission time and bandwidth, and thus helps to improve synchronization efficiency. Furthermore, by constructing a hash table, it is helpful to speed up the comparison of the first source-end verification value and the first target-end verification value, thereby determining the incremental data more quickly and further improving the synchronization efficiency.

[0036] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0038] Figure 1 is a schematic diagram of a data synchronization method of a database system according to an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of a data synchronization method of a database system according to another embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of a data synchronization method of a database system according to another embodiment of the present invention;

[0041] Figure 4 is a schematic diagram of a data synchronization method of a database system according to another embodiment of the present invention;

[0042] Figure 5 is a schematic diagram of target-side database segmentation according to an embodiment of the present invention;

[0043] Figure 6 is a schematic diagram of a data synchronization method of a database system according to another embodiment of the present invention;

[0044] Figure 7 is a schematic diagram of an information set and a hash table of a source database according to an embodiment of the present invention;

[0045] Figure 8 is a schematic diagram of a data synchronization method of a database system according to another embodiment of the present invention;

[0046] Fig. 9 is a schematic diagram of a data synchronization method of a database system according to another embodiment of the present invention;

[0047] Fig.10 is a schematic diagram of a computer device according to an embodiment of the present invention;

[0048] Fig.11 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention;

[0049] Fig.12 is a schematic diagram of a computer program product according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] It should be understood by those skilled in the art that the embodiments described below are only some embodiments of the present invention, rather than all embodiments of the present invention, and these embodiments are intended to explain the technical principles of the present invention, rather than to limit the protection scope of the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present invention.

[0051] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or used in combination with these instruction execution systems, devices or equipment.

[0052] The flow chart provided by the present invention is not intended to indicate that the operation of the method will be performed in any particular order, or all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical thinking provided by the present embodiment method, additional changes can be made to the above method.

[0053] like Figure 1 As shown, in one embodiment, the database system includes a source database and a target database, the source database is the party that provides synchronization data, and the target database is the party that receives synchronization data. In fault reconstruction, the source database is the surviving database, and the target database is the faulty database to be reconstructed. The data synchronization method of the database system generally includes:

[0054] Step S101, read source data blocks from data files in a source database according to a preset size. Specifically, the preset size may be 4kb, 8kb, etc.

[0055] Step S102, calculate the first source check value of the source data block according to the first preset algorithm. Specifically, the first preset algorithm can be a simple byte sum algorithm, a cyclic redundancy check algorithm, etc. The algorithm for calculating the check value of the data block is well known to those skilled in the art and will not be described in detail here.

[0056] Step S103, check whether the first source-end check value is mapped in the hash table, if so, execute step S104, if not, execute step S105. The hash table is constructed based on the data block information of the target-end database, and the data block information includes the unique number of each data block divided by the target-end database according to a preset size and the first target-end check value calculated according to the first preset algorithm. Specifically, the first target-end check value is converted into a key of the hash table, that is, the first target-end check value is converted into a key of the hash table using a hash algorithm. The value of the hash table can be the unique number in the data block information, or it can be a mark reassigned to each data block information after the source-end database receives the data block information.

[0057] Step S104, determine that the source data block is non-incremental data that does not need to be synchronized to the target database. If the first source check value is mapped in the hash table, it means that the first source check value is the same as the first target check value stored in the hash table mapping position, that is, the current source data block has the same target data block, that is, the data of the current source data block also exists in the target database, so it does not need to be synchronized to the target database.

[0058] Step S105, determine that the source data block is incremental data that needs to be synchronized to the target database. If the first source check value does not have a mapping in the hash table, it means that the first source check value does not have the same first target check value, that is, the current source data block does not have the same target data block, that is, the target database does not have the data of the current source data block, so it needs to be synchronized to the target database.

[0059] Step S106, sending the source data block to the target database, that is, synchronizing the source data block to the target database.

[0060] In the scheme of this embodiment, the source data block is read from the data file of the source database according to the preset size, the first source check value of the source data block is calculated according to the first preset algorithm, and it is checked whether the first source check value is mapped in the hash table. If not, it is determined that the source data block is incremental data that needs to be synchronized to the target database, and the source data block is sent to the target database. The hash table is constructed according to the data block information of the target database, and the data block information includes the unique number of each data block divided by the target database according to the preset size and the first target check value calculated according to the first preset algorithm. In other words, the source data block that the target database does not have can be compared according to the mapping of the first source check value of the source data block and the first target check value of the target data block in the hash table, that is, the incremental data of the source database relative to the target database can be compared. During synchronization, only the source data block as incremental data needs to be synchronized to the target database, which helps to reduce the amount of data that needs to be transmitted, saves transmission time and bandwidth, and thus helps to improve synchronization efficiency. Furthermore, by constructing a hash table, it is helpful to speed up the comparison of the first source-end verification value and the first target-end verification value, thereby determining the incremental data more quickly and further improving the synchronization efficiency.

[0061] like Figure 2 As shown, in one embodiment, the data synchronization method of the database system generally includes:

[0062] Step S201 : reading source data blocks from data files of a source database according to a preset size.

[0063] Step S202: Calculate a first source check value of a source data block according to a first preset algorithm.

[0064] Step S203, check whether the first source verification value is mapped in the hash table, if so, execute step S204, if not, execute step S208.

[0065] Step S204, searching the information set for the corresponding first target-side check value according to the mapping value. The information set is used to record the data block information from the target-side database. Specifically, after the source-side database receives the data block information from the target-side database, it uses all the data block information to construct an information set, that is, a data table storing the data block information. The information set records the value of the hash table, that is, the unique number or the mark reset by the source-side database for each data block information. In this way, the mapping is found in the hash table, and the value of the mapping position, that is, the mapping value, is read, and the first target-side check value can be found in the information set according to the mapping value.

[0066] Step S205, comparing the first target-end verification value with the first source-end verification value. After the first target-end verification value is found in the information set, the first target-end verification value is compared with the first source-end verification value.

[0067] Step S206, determine whether the first source check value and the first target check value are the same, if yes, execute step S207, if no, execute step S208. Because different check values ​​may calculate the same mapping position in the hash table, even if there is a mapping, the original first source check value and the first target check value are compared to confirm whether they are the same.

[0068] Step S207, determine that the source data block is non-incremental data that does not need to be synchronized to the target database. If the first source check value is the same as the first target check value, that is, the current source data block has the same target data block, that is, the data of the current source data block also exists in the target database, so it does not need to be synchronized to the target database.

[0069] Step S208, determine that the source data block is incremental data that needs to be synchronized to the target database. If the first source check value does not have a mapping in the hash table, it means that the first source check value does not have the same first target check value, that is, the current source data block does not have the same target data block, that is, the target database does not have the data of the current source data block, so it needs to be synchronized to the target database.

[0070] Alternatively, if the comparison result of the first source-end verification value and the first target-end verification value is different, that is, the current source-end data block does not have the same target-end data block, that is, the data of the current source-end data block does not exist in the target-end database, so it needs to be synchronized to the target-end database.

[0071] Step S209: Send the source data block to the target database, that is, synchronize the source data block to the target database.

[0072] In the scheme of this embodiment, under the condition that the first source-end check value is mapped in the hash table, the corresponding first target-end check value is searched in the information set according to the mapping value and the first target-end check value is compared with the first source-end check value to determine whether the first source-end check value and the first target-end check value are the same. If so, the source-end data block is determined to be non-incremental data that does not need to be synchronized to the target-end database. If not, the source-end data block is determined to be incremental data that needs to be synchronized to the target-end database. Because different check values ​​may calculate the same mapping position in the hash table, when there is a mapping, the original first target-end check value and the first source-end check value are found and compared, and based on the comparison result, it is further confirmed whether the current source-end data block is incremental data to reduce the occurrence of misjudgment.

[0073] For steps not described in detail in this embodiment, refer to the above description.

[0074] like Figure 3 As shown, in one embodiment, the data synchronization method of the database system generally includes:

[0075] Step S301, reading source data blocks from data files of a source database according to a preset size.

[0076] Step S302: Calculate a first source check value of a source data block according to a first preset algorithm.

[0077] Step S303, check whether the first source-end verification value is mapped in the hash table, if so, execute step S304, if not, execute step S310.

[0078] Step S304: searching the information set for the corresponding first target end verification value according to the mapping value.

[0079] Step S305, comparing the first target-end verification value and the first source-end verification value.

[0080] Step S306, determining whether the first source-end verification value and the first target-end verification value are the same, if so, executing step S307, if not, executing step S308.

[0081] Step S307: determine that the source-side data block is non-incremental data that does not need to be synchronized to the target-side database.

[0082] Step S308, check whether there is a conflicting mapping value in the mapping position of the hash table. If so, execute step S309, if not, execute step S310. Specifically, in the process of constructing the hash table in the source database, different first target end check values ​​may calculate the same position in the hash table, that is, there is a hash conflict. At this time, a linked list is established at the position, and the subsequent hash values ​​are stored in the linked list. This step means that all mapping values ​​at the mapping position need to be traversed and checked.

[0083] Therefore, when the first source-end check value and the first target-end check value found for the first time are different, it cannot be determined that the source-end data block is incremental data, because there may be other values ​​at the same mapping position, thereby finding other first target-end check values ​​that may be the same as the first source-end check value in the information set.

[0084] Step S309, search the corresponding first target end check value in the information set according to the conflicting mapping value and compare the first target end check value with the first source end check value, and then proceed to step S306. If there is a conflicting mapping value in the hash table, find a new first target end check value in the information set according to the conflicting mapping value and compare the first target end check value with the first source end check value.

[0085] Step S310, determine that the source data block is incremental data that needs to be synchronized to the target database. If all mapping values ​​of the first source check value in the hash table mapping position are traversed, and the comparison results of the first target check value found in the information set and the first source check value are different, it is determined that the source data block needs to be synchronized to the target database.

[0086] Step S311, sending the source data block to the target database, that is, synchronizing the source data block to the target database.

[0087] In the scheme of the present embodiment, when the first source-end check value and the first target-end check value are different, it is checked whether there are conflicting mapping values ​​in the mapping position of the hash table. If there are conflicting mapping values, the corresponding first target-end check value is searched in the information set according to the conflicting mapping value and the first target-end check value is compared with the first source-end check value. That is to say, when there are multiple mapping values ​​at the same mapping position, all the first target-end check values ​​found are compared with the first source-end check value. Only when the comparison results are different is the source-end data block determined to be incremental data, thereby reducing the occurrence of misjudgment.

[0088] For steps not described in detail in this embodiment, refer to the above description.

[0089] like Figure 4 As shown, in one embodiment, the data synchronization method of the database system generally includes:

[0090] Step S401, divide the data file of the target database into data blocks according to the preset size and mark them with unique numbers in sequence. Specifically, the unique number can be a number that increases from 1, as shown in FIG. Figure 5 The figure shows an example of dividing the data files of a target database into blocks and numbering them starting from 1.

[0091] Step S402, read and calculate the first target end check value and the second target end check value of each data block in sequence. Specifically, two different check values ​​of the same data block are calculated using two different algorithms. That is, the first target end check value is calculated using the first preset algorithm, and the second target end check value is calculated using the second preset algorithm.

[0092] Step S403, sending the data block information consisting of the unique number, the first target end check value and the second target end check value to the source end database. Figure 5 As shown, the number 3 of the data block 3 and the first target-end check value and the second target-end check value are the data block information to be sent to the source-end database.

[0093] like Figure 6As shown, in this embodiment, the data synchronization method of the database system generally includes:

[0094] Step S601, receiving data block information from a target database. Specifically, the source database receives data block information from the target database, that is, a unique number, a first target check value, and a second target check value of each target data block.

[0095] Step S602: construct an information set based on the data block information. Figure 7 As shown, the data block information from the target database is stored in the table according to the corresponding relationship. At the same time, an index subscript is set for each set of data block information, that is, Figure 7 The first column is numbered starting with 0.

[0096] Step S603: construct a hash table according to the data block information, wherein the first target end check value is converted into a key of the hash table. Figure 7 As shown, that is, each first target end check value is calculated according to the hash algorithm to obtain the mapping position in the hash table ( Figure 7 The column represented by Key in the table is then stored as the value in the corresponding position ( Figure 7 In the process of constructing a hash table, if a hash conflict occurs, the conflicting values ​​are stored in the same position in the form of a conflict linked list, such as Figure 7 As shown, the first target end check values ​​corresponding to index subscripts 43 and 44 calculate the same position, so 43 and 44 are stored in the conflict linked list.

[0097] like Figure 8 As shown, in this embodiment, the data synchronization method of the database system generally includes:

[0098] Step S801, reading source data blocks from data files of a source database according to a preset size.

[0099] Step S802: Calculate a first source check value of a source data block according to a first preset algorithm.

[0100] Step S803, check whether the first source-end verification value is mapped in the hash table, if so, execute step S804, if not, execute step S814.

[0101] Step S804: searching the information set for the corresponding first target end verification value according to the mapping value.

[0102] Step S805, comparing the first target-end verification value and the first source-end verification value.

[0103] Step S806, determine whether the first source-end verification value and the first target-end verification value are the same, if so, execute step S807, if not, execute step S812.

[0104] Step S807: Calculate the second source check value of the source data block according to the second preset algorithm, that is, calculate the second source check value of the source data block using the same second preset algorithm as that used to calculate the second target check value.

[0105] Step S808: Search the information set for a corresponding second target end verification value according to the mapping value. The mapping value represents the same mapping value as the corresponding first target end verification value.

[0106] Step S809, comparing the second target-end verification value and the second source-end verification value.

[0107] Step S810, determining whether the second source-end verification value is the same as the second target-end verification value, if so, executing step S811, if not, executing step S812.

[0108] Step S811, determine that the source data block is non-incremental data that does not need to be synchronized to the target database. Specifically, the first source check value is the same as the first target check value, and the second source check value is the same as the second target check value, and the source data block is determined to be non-incremental data that does not need to be synchronized to the target database.

[0109] Step S812, check whether there is a conflicting mapping value in the mapping position in the hash table, if so, execute step S813, if not, execute step S814.

[0110] Step S813, searching the information set for the corresponding first target-end verification value according to the conflicting mapping value and comparing the first target-end verification value with the first source-end verification value, and then proceeding to step S806.

[0111] Step S814, determine that the source data block is incremental data that needs to be synchronized to the target database. Specifically, if the first target check value and the second target check value corresponding to all mapping values ​​of the traversal mapping position do not have data block information where the first source check value is the same as the first target check value, and the second source check value is the same as the second target check value, then it is determined to be incremental data.

[0112] Step S815: Send the source data block to the target database, that is, synchronize the source data block to the target database.

[0113] Combination Figure 7As shown, the method of this embodiment is exemplarily described. When a source data block is read, the first source check value of the source data block is calculated. If the first source check value does not exist in the hash table, it is determined to be incremental data, and the next source data block is read for judgment.

[0114] If the first source checksum value is mapped in the hash table:

[0115] If mapped to Figure 7 k0 in , then find the corresponding first target end check value in the information set according to mapping value 2 (the index subscript of the first column of the information set), compare whether the first target end check value is the same as the first source end check value, and then,

[0116] If the first source-end check value is different from the first target-end check value, and there is no conflict mapping value at position k0 of the hash table, it is determined to be incremental data, and the next source-end data block is read for judgment;

[0117] If the first source check value is the same as the first target check value, calculate the second source check value of the source data block according to the second preset algorithm, find the corresponding second target check value in the information set according to the mapping value 2 (the index subscript of the first column of the information set), compare the second target check value with the second source check value to see if they are the same, and then,

[0118] If the second source check value is different from the second target check value, and there is no conflict mapping value at position k0 of the hash table, it is determined to be incremental data, and the next source data block is read for judgment;

[0119] If the second source-end check value is the same as the second target-end check value, it is determined to be non-incremental data, and the next source-end data block is read for determination.

[0120] If mapped to Figure 7 k4 in the information set, then find the corresponding first target end check value in the information set according to the mapping value 43 (the index subscript of the first column of the information set), compare whether the first target end check value is the same as the first source end check value, and then,

[0121] If the first source-end check value is different from the first target-end check value corresponding to 43, and there is a conflicting mapping value 44 at position k4 of the hash table, then the corresponding first target-end check value is found in the information set according to the conflicting mapping value 44, and then compared with the first source-end check value. If they are still different and there are no other conflicting mapping values, it is determined to be incremental data, and the next source-end data block is read for judgment;

[0122] If the first target-end check value corresponding to 44 is the same as the first source-end check value, the second source-end check value of the source-end data block is calculated according to the second preset algorithm, and the corresponding second target-end check value is found in the information set according to the mapping value 44, and the second target-end check value corresponding to 44 is compared to see whether they are the same; if the second source-end check value and the second target-end check value corresponding to 44 are not the same, and there are no other conflicting mapping values ​​at the k4 position of the hash table, it is determined to be incremental data, and the next source-end data block is read for judgment; if the second source-end check value and the second target-end check value corresponding to 44 are the same, it is determined to be non-incremental data, and the next source-end data block is read for judgment.

[0123] If the first source-end check value is the same as the first target-end check value corresponding to 43, a second source-end check value of the source-end data block is calculated according to a second preset algorithm, and a corresponding second target-end check value is found in the information set according to the mapping value 43, and the second target-end check value corresponding to 43 is compared to see whether it is the same as the second source-end check value. If the second source-end check value is the same as the second target-end check value corresponding to 43, it is determined to be non-incremental data, and the next source-end data block is read for judgment;

[0124] If the second source-end check value is different from the second target-end check value corresponding to 43, and there is a conflicting mapping value 44 at position k4 of the hash table, then the corresponding first target-end check value is found in the information set according to the conflicting mapping value 44, and then compared with the first source-end check value. If they are different and there are no other conflicting mapping values, it is determined to be incremental data, and the next source-end data block is read for judgment; if the first target-end check value corresponding to 44 is the same as the first source-end check value, the second source-end check value of the source data block is calculated according to the second preset algorithm, and the corresponding second target-end check value is found in the information set according to the mapping value 44, and the second target-end check value corresponding to 44 is compared to see if they are the same. If the second source-end check value is different from the second target-end check value corresponding to 44, and there are no other conflicting mapping values ​​at position k4 of the hash table, it is determined to be incremental data, and the next source-end data block is read for judgment; if the second source-end check value is the same as the second target-end check value corresponding to 44, it is determined to be non-incremental data, and the next source-end data block is read for judgment.

[0125] In the scheme of this embodiment, a three-layer confirmation method is used to determine whether the source data block and the target data block are the same through hash mapping, comparison of the first source verification value and the first target verification value, and comparison of the second source verification value and the second target verification value, thereby improving the comparison efficiency and effectively avoiding the occurrence of misjudgment.

[0126] like Fig. 9 As shown, in one embodiment, the data synchronization method of the database system includes:

[0127] Step S901, create a temporary file in the target database, that is, create a temporary storage location.

[0128] Step S902, receiving incremental data blocks from the source database and storing them in a temporary file. After the source database determines the source data blocks belonging to the incremental data, the source data blocks are sent to the target database, which receives them and stores them in a temporary file.

[0129] Step S903, receiving the unique number corresponding to the non-incremental data block from the source database and copying the corresponding data block of the target database to a temporary file. After the source database determines the source data block that belongs to the non-incremental data, the unique number of the same target data block corresponding to the source data block is sent to the target database, and the target database copies the corresponding data block to the temporary file after receiving it.

[0130] Step S904: After detecting that all information from the source database has been processed, the original data file of the target database is deleted and the temporary file is renamed to the name of the original data file. Specifically, all source data blocks of the source database are determined to be completed, and all incremental or non-incremental related data or information have been sent to the target database, and the target database has also been processed, indicating that the data stored in the temporary file is consistent with the source database, then the original data file of the target database is deleted and the temporary file is renamed to the name of the original data file.

[0131] By enabling the target database to receive unique numbers corresponding to incremental data blocks and non-incremental data, it helps to determine whether all target data blocks have corresponding source data blocks, thereby ensuring the consistency of data synchronization.

[0132] This embodiment also provides a computer device and a computer-readable storage medium. Fig.10 is a schematic diagram of a computer device 10 according to one embodiment of the present invention. Fig.11 is a schematic diagram of a computer-readable storage medium 20 according to one embodiment of the present invention.

[0133] The computer device 10 may include a memory 110, a processor 120, and a computer executable program 11 stored in the memory 110 and running on the processor 120, and the processor 120 implements the data synchronization method of the database system of any of the above embodiments when executing the computer executable program 11.

[0134] The computer-readable storage medium 20 stores a computer-executable program 11 thereon. When the computer-executable program 11 is executed by a processor, the method for synchronizing data in a database system according to any of the above-mentioned embodiments can be implemented.

[0135] This embodiment also provides a computer program product. Fig.12 1 is a schematic diagram of a computer program product 30 according to an embodiment of the present invention. The computer program product 30 includes a computer executable program 11, and when the computer executable program 11 is executed by a processor 120, it can implement any of the above-mentioned methods for synchronizing data in a database system.

[0136] Specifically, the computer executable program 11 for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, computer instructions, computer-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages.

[0137] For the purpose of the description of the present embodiment, the computer-readable storage medium 20 can be any device that can contain, store, communicate, propagate or transmit a program for use with an instruction execution system, device or equipment or in conjunction with these instruction execution systems, devices or equipment. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion (electronic device) with one or more wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium 20 can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting or processing in other suitable ways as necessary, and then stored in a computer memory.

[0138] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0139] The computer device 10 may be, for example, a server, a desktop computer, a notebook computer, a tablet computer, or a smart phone. In some examples, the computer device 10 may be a cloud acquisition node. The computer device 10 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, a program module may include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 10 may be implemented in a distributed cloud acquisition environment where remote processing devices linked via a communication network perform tasks. In a distributed cloud acquisition environment, program modules may be located on a local or remote acquisition system storage medium including a storage device.

[0140] The computer device 10 may include a processor 120 adapted to execute stored instructions, and a memory 110 that provides temporary storage space for the operation of the instructions during operation. The processor 120 may be a single-core processor, a multi-core processor, an acquisition cluster, or any number of other configurations. The memory 110 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0141] The processor 120 may be connected to an I / O interface (input / output interface) suitable for connecting the computer device 10 to one or more I / O devices (input / output devices) via a system interconnect (e.g., PCI, PCI-Express, etc.). The I / O devices may include, for example, a keyboard and a pointing device, wherein the pointing device may include a touch pad or a touch screen, etc. The I / O devices may be built-in components of the computer device 10, or may be devices externally connected to an acquisition device.

[0142] Processor 120 can also be linked to a display interface suitable for connecting computer device 10 to a display device through a system interconnection. Display device can include a display screen as a built-in component of computer device 10. Display device can also include a computer monitor, a television or a projector, etc., which are externally connected to computer device 10. In addition, a network interface controller (NIC) can be suitable for connecting computer device 10 to a network through a system interconnection. In some embodiments, NIC can use any suitable interface or protocol (such as an Internet small computer system interface, etc.) to transmit data. The network can be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN) or the Internet, etc. A remote device can be connected to a computer device through a network.

[0143] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A data synchronization method for a database system, comprising: Read source data blocks from data files in the source database according to a preset size; Calculate a first source check value of the source data block according to a first preset algorithm; Check whether the first source-end check value is mapped in the hash table; if not, determine that the source-end data block is incremental data that needs to be synchronized to the target-end database, and send the source-end data block to the target-end database; The hash table is constructed according to data block information of the target database, and the data block information includes a unique number of each data block divided by the target database according to the preset size and a first target check value calculated according to the first preset algorithm.

2. The data synchronization method of a database system according to claim 1, wherein: After the step of checking whether the first source check value is mapped in the hash table, the following steps are included: If so, searching for a corresponding first target-end check value in an information set according to the mapping value and comparing the first target-end check value with the first source-end check value, the information set being used to record data block information from the target-end database; Determine whether the first source-side verification value and the first target-side verification value are the same. If so, determine that the source-side data block is non-incremental data that does not need to be synchronized to the target-side database. If not, determine that the source-side data block is incremental data that needs to be synchronized to the target-side database.

3. The data synchronization method of a database system according to claim 2, wherein: The data block information also includes a second target end check value calculated according to a second preset algorithm; If the first source-end check value is the same as the first target-end check value, the data synchronization method of the database system further includes: Calculating a second source check value of the source data block according to the second preset algorithm; Searching for a corresponding second target-end verification value in the information set according to the mapping value and comparing the second target-end verification value with the second source-end verification value; Determine whether the second source-side verification value and the second target-side verification value are the same. If so, determine that the source-side data block is non-incremental data that does not need to be synchronized to the target-side database. If not, determine that the source-side data block is incremental data that needs to be synchronized to the target-side database.

4. The method for synchronizing data in a database system according to claim 3, wherein: The data synchronization method of the database system comprises: Dividing the data files of the target database into data blocks according to the preset size and marking them with unique numbers in sequence; Read and calculate the first target-end check value and the second target-end check value of each of the data blocks in sequence; The data block information consisting of the unique number, the first target-end verification value and the second target-end verification value is sent to the source-end database.

5. The method for synchronizing data in a database system according to claim 4, wherein: The data synchronization method of the database system comprises: Receiving data block information from the target database; Constructing the information set according to the data block information; The hash table is constructed according to the data block information, wherein the first target-end check value is converted into a key of the hash table.

6. The method for synchronizing data in a database system according to claim 2, wherein: If the first source-end check value is different from the first target-end check value, the data synchronization method of the database system further includes: Check whether there are conflicting mapping values ​​in the mapping position of the hash table. If so, search the corresponding first target-end verification value in the information set according to the conflicting mapping value and compare the first target-end verification value with the first source-end verification value, and execute the step of determining whether the first source-end verification value and the first target-end verification value are the same. If not, determine that the source-end data block is incremental data that needs to be synchronized to the target-end database.

7. The data synchronization method of a database system according to claim 1, wherein: The data synchronization method of the database system comprises: Creating a temporary file in the target database; Receiving incremental data blocks from the source database and storing them in the temporary file; Receiving a unique number corresponding to a non-incremental data block from the source database and copying the data block corresponding to the target database to the temporary file; It is detected that all information processing from the source database is completed, the original data file of the target database is deleted and the temporary file is renamed to the name of the original data file.

8. A computer device comprising a memory, a processor, and a computer executable program stored in the memory and running on the processor, wherein the processor implements the data synchronization method of the database system according to any one of claims 1 to 7 when executing the computer executable program.

9. A computer-readable storage medium having a computer-executable program stored thereon, wherein the computer-executable program, when executed by a processor, implements the data synchronization method of a database system according to any one of claims 1 to 7.

10. A computer program product, comprising a computer executable program, wherein when the computer executable program is executed by a processor, the method for synchronizing data in a database system according to any one of claims 1 to 7 is implemented.

Images