Data verification methods and related products

Abstract

The present invention provides a data verification method for incremental reconstruction of a database and related products, including: dividing a to-be-synchronized file and a synchronization reference file into a plurality of to-be-synchronized data blocks and a plurality of synchronization reference data blocks respectively according to the same division method; uniformly setting the page structure parameters of the to-be-synchronized data blocks and the synchronization reference data blocks; constructing a verification information set for the to-be-synchronized data blocks; obtaining synchronization reference data blocks block by block, and matching the obtained synchronization reference data blocks with the verification information set; if the verification information set does not contain verification information matching the obtained synchronization reference data blocks, then judging that the obtained synchronization reference data blocks belong to incremental data. The data verification method for incremental reconstruction of a database of the present invention can effectively limit the misjudgment of incremental data, further limit the increase in network broadband load caused by misjudged incremental data, and improve the synchronization performance of the database.

Description

Technical Field

[0001] The present invention relates to the technical field of database synchronization, and in particular to a data verification method and related products. Background Art

[0002] With the rapid development of information technology, database management systems are being used more and more widely, and data synchronization and backup have become key technologies to ensure data consistency and high availability. In database systems, especially in distributed databases or scenarios where frequent data migration is required, incremental reconstruction of the database has become an important technical means.

[0003] At present, in the prior art, when incrementally rebuilding a faulty database to achieve data synchronization through an "incremental mining algorithm", the same table in the data files of different databases (primary node, backup node) may have the same data content actually stored in the primary node and the backup node, but the checksum calculated for the same page (page, or data block) may be inconsistent. This situation usually occurs due to inconsistent page structure parameters before synchronization. In actual applications, pages with inconsistent checksums are mistakenly judged as incremental data. These misjudged incremental data need to be transmitted to the faulty database through the network to achieve data synchronization. However, these misjudged incremental data increase the load on the network bandwidth, especially when the network is limited, which has a great impact on the synchronization performance, resulting in a decrease in the synchronization performance of the database. Summary of the invention

[0004] An object of the present invention is to provide a data verification method for incremental reconstruction of a database and related products that can overcome at least one of the above-mentioned defects in the prior art.

[0005] A further object of the present invention is to avoid the situation where the actual stored data content is the same but the checksum is different due to the inconsistent page structure of the data block to be synchronized and the synchronization reference data block, effectively limit the misjudgment of incremental data, limit the increase in network broadband load caused by misjudged incremental data, and improve the synchronization performance of the database.

[0006] In particular, the present invention provides a data verification method for incremental reconstruction of a database, comprising:

[0007] Dividing the to-be-synchronized file and the synchronization reference file into a plurality of to-be-synchronized data blocks and a plurality of synchronization reference data blocks respectively according to the same division method;

[0008] Unified setting of page structure parameters of the data blocks to be synchronized and the synchronization reference data blocks;

[0009] Construct a set of verification information for the data blocks to be synchronized;

[0010] Obtaining synchronization reference data blocks one by one, and matching the obtained synchronization reference data blocks with the verification information set;

[0011] If the verification information set does not contain verification information that matches the acquired synchronization reference data block, it is determined that the acquired synchronization reference data block belongs to incremental data.

[0012] Furthermore, the page structure parameter includes a tuple insertion status flag (a bit in the T_infomask field indicating that T_xmin has been submitted), and the tuple insertion status flag is used to indicate whether the last transaction that inserted the tuple has been submitted;

[0013] The steps of uniformly setting the page structure parameters of the data block to be synchronized and the synchronization reference data block include:

[0014] Set the tuple insertion status flag of the data block to be synchronized to the initial value;

[0015] Set the tuple insertion status flag of the synchronization reference data block to the initial value.

[0016] Furthermore, the page structure parameters also include a page header checksum field (Pd_checksum), which is used to store the checksum of the data block;

[0017] After the step of setting the tuple insertion status flag of the data block to be synchronized to the initial value, the step of uniformly setting the page structure parameters of the data block to be synchronized also includes:

[0018] The page header check field of the data block to be synchronized is set to the initial value; and,

[0019] After the step of setting the tuple insertion state flag of the synchronization reference data block to an initial value, the step of uniformly setting the page structure parameters of the synchronization reference data block further includes:

[0020] Set the page header checksum field of the synchronization reference data block to the initial value.

[0021] Furthermore, the step of constructing a verification information set of the data block to be synchronized includes:

[0022] Calculate a first checksum to be synchronized and a second checksum to be synchronized for each data block to be synchronized;

[0023] Number each data block to be synchronized;

[0024] Constructing a check information set with an index subscript according to the number of each data block to be synchronized, each first checksum to be synchronized, and each second checksum to be synchronized;

[0025] A hash table is constructed according to each first checksum to be synchronized and each index subscript corresponding thereto, wherein each first checksum to be synchronized is a key value of the hash table, and each index subscript is a hash value of the hash table; and,

[0026] The step of matching the acquired synchronization reference data block with the verification information set includes:

[0027] Calculating a first synchronization reference checksum of the acquired synchronization reference data block;

[0028] Find out whether the calculated first synchronization reference checksum is mapped in the hash table;

[0029] If not, it is determined that there is no verification information in the verification information set that matches the acquired synchronization reference data block.

[0030] Further, in the case where the calculated first synchronization reference checksum does not have a mapping in the hash table, the step of matching the acquired synchronization reference data block with the check information set further includes:

[0031] Retrieving the first to-be-synchronized checksum corresponding to the calculated first synchronization reference checksum from the check information set using the hash value;

[0032] Determining whether the calculated first synchronization reference checksum and the retrieved first to-be-synchronized checksum are the same;

[0033] If not, searching the conflict linked list of the hash table for a first checksum to be synchronized corresponding to the calculated first synchronization reference checksum;

[0034] Comparing the calculated first synchronization reference checksum and the searched first to-be-synchronized checksum to see if they are the same;

[0035] When the calculated first synchronization reference checksum is different from the searched first to-be-synchronized checksum, a step of determining that there is no check information in the check information set that matches the acquired synchronization reference data block is performed.

[0036] Further, when the calculated first synchronization reference checksum and the retrieved first checksum to be synchronized are the same, and when the calculated first synchronization reference checksum and the searched first checksum to be synchronized are the same, the step of matching the acquired synchronization reference data block with the check information set further includes:

[0037] Calculating a second synchronization reference checksum of the acquired synchronization reference data block;

[0038] Retrieving the second to-be-synchronized checksum corresponding to the calculated second synchronization reference checksum from the check information set using the hash value;

[0039] Determining whether the calculated second synchronization reference checksum is the same as the retrieved second to-be-synchronized checksum;

[0040] If not, a step of searching the conflict linked list of the hash table for a first checksum to be synchronized corresponding to the calculated first synchronization reference checksum is executed.

[0041] Further, when the calculated second synchronization reference checksum is the same as the retrieved second to-be-synchronized checksum, the step of matching the acquired synchronization reference data block with the check information set further includes:

[0042] Determining whether there is verification information in the verification information set that matches the acquired synchronization reference data block; and,

[0043] The data verification method for incremental reconstruction of the database also includes:

[0044] If the verification information set contains verification information that matches the acquired synchronization reference data block, it is determined that the acquired synchronization reference data block belongs to matching data.

[0045] Furthermore, the step of dividing the synchronization reference file into a plurality of synchronization reference data blocks comprises:

[0046] Dividing the synchronization reference file into a plurality of synchronization data blocks having the same size as the data block to be synchronized;

[0047] Copy the synchronization data block to obtain the synchronization reference data block.

[0048] In particular, the present invention further provides a machine-readable storage medium on which a machine executable program is stored. When the machine executable program is executed by a processor, the data verification method for incremental reconstruction of a database described above is implemented.

[0049] In particular, the present invention also provides a computer device, including a memory, a processor, and a machine executable program stored in the memory and running on the processor, and the processor implements the above-mentioned data verification method for incremental reconstruction of the database when executing the machine executable program.

[0050] The data verification method for incremental reconstruction of a database of the present invention performs unified setting of the page structure parameters of the data block to be synchronized and the synchronization reference data block before the data block to be synchronized and the synchronization reference data block are verified and calculated. Therefore, the data verification method for incremental reconstruction of a database of the present invention can avoid the situation where the actual stored data content is the same but the verification sum is different due to the inconsistent page structure of the data block to be synchronized and the synchronization reference data block, and effectively limits the misjudgment of incremental data, further limits the increase of network broadband load caused by the misjudged incremental data, and improves the synchronization performance of the database.

[0051] The machine-readable storage medium of the present invention can realize the data verification method for incremental reconstruction of the above-mentioned database. Therefore, the machine-readable storage medium of the present invention also has the beneficial technical effects that can be achieved by the data verification method for incremental reconstruction of the above-mentioned database.

[0052] The computer device of the present invention can implement the above-mentioned data verification method for incremental reconstruction of a database. Therefore, the computer device of the present invention also has the beneficial technical effects that can be achieved by the above-mentioned data verification method for incremental reconstruction of a database.

[0053] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0055] Figure 1 is a flow chart of a data verification method for incremental reconstruction of a database according to an embodiment of the present invention;

[0056] Figure 2 is a flow chart of a data verification method for incremental reconstruction of a database according to another embodiment of the present invention;

[0057] Figure 3 It is a flow chart of dividing a synchronization reference file into a plurality of synchronization reference data blocks in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention;

[0058] Figure 4 It is a schematic diagram of a frame structure of a file to be synchronized or a synchronization reference file in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention;

[0059] Figure 5 It is a schematic diagram of a block structure of a data block to be synchronized or a synchronization reference data block in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention;

[0060] Figure 6 Schematic diagram of the frame structure of the verification information set in the data verification method for incremental reconstruction of a database according to an embodiment of the present invention

[0061] Figure 7is a schematic diagram of a hash table, a verification information set, and a mapping organization form in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention;

[0062] Figure 8 is a schematic diagram of a block structure of a machine-readable storage medium according to an embodiment of the present invention;

[0063] Fig. 9 It is a schematic diagram of the frame structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In the description of the present embodiment, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features, that is, include one or more of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. When a feature "includes or contains" one or some of the features it covers, unless otherwise specifically described, this indicates that other features are not excluded and other features may be further included.

[0065] Unless otherwise defined, all terms (including technical terms and scientific terms) used in the description of this embodiment have the same meanings as commonly understood by ordinary technicians in the technical field to which this application belongs.

[0066] In the description of the present embodiment, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0067] Figure 1 is a flow chart of a data verification method for incremental reconstruction of a database according to an embodiment of the present invention; Figure 4 It is a schematic diagram of a frame structure of a file to be synchronized or a synchronization reference file in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention; Figure 5 1 is a schematic diagram of a block structure of a data block to be synchronized or a synchronization reference data block in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention. Figure 1 In this embodiment, the data verification method for incremental reconstruction of the database may include:

[0068] Step S101 : dividing the to-be-synchronized file and the synchronization reference file into a plurality of to-be-synchronized data blocks and a plurality of synchronization reference data blocks respectively according to the same division method.

[0069] Step S102, uniformly setting page structure parameters of the data block to be synchronized and the synchronization reference data block;

[0070] Step S103, constructing a verification information set of the data block to be synchronized;

[0071] Step S104, obtaining synchronization reference data blocks one by one, and matching the obtained synchronization reference data blocks with the verification information set;

[0072] Step S105: If the verification information set does not contain verification information that matches the acquired synchronization reference data block, it is determined that the acquired synchronization reference data block belongs to incremental data.

[0073] Reference Figure 5 The data verification method for incremental reconstruction of a database of the present embodiment, because it performs unified setting of the page structure parameters of the data blocks to be synchronized and the synchronization reference data blocks before verifying and calculating the data blocks to be synchronized and the synchronization reference data blocks. The data verification method for incremental reconstruction of a database of the present invention, because it performs unified setting of the page structure parameters of the data blocks to be synchronized and the synchronization reference data blocks before verifying and calculating the data blocks to be synchronized and the synchronization reference data blocks. Therefore, the data verification method for incremental reconstruction of a database of the present embodiment can avoid the situation where the actual stored data content is the same but the checksum is different due to the inconsistency of the page structure parameters of the data blocks to be synchronized and the synchronization reference data blocks, and effectively limits the misjudgment of incremental data, further limits the increase in network broadband load caused by the misjudged incremental data, and improves the synchronization performance of the database.

[0074] Reference Figure 4 In this embodiment, in step S101, the file to be synchronized is divided into a plurality of data blocks to be synchronized, which may be divided into a plurality of data blocks to be synchronized according to the page size of the database (for example, according to the page size of the database to be synchronized).

[0075] In this embodiment, after the step of matching the acquired synchronization reference data block with the verification information set, the data verification method for incremental reconstruction of the database may further include:

[0076] If the verification information set contains verification information that matches the acquired synchronization reference data block, it is determined that the acquired synchronization reference data block belongs to matching data.

[0077] In addition, step S102 may include steps S202 to S205 in the following embodiments. Step S103 may include steps S206 to S209 in the following embodiments. Step S104 may include steps S207 to S220 in the following embodiments.

[0078] Figure 2 is a flow chart of a data verification method for incremental reconstruction of a database according to another embodiment of the present invention; Figure 6 It is a schematic diagram of a block structure of a verification information set in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention; Figure 7 is a schematic diagram of a hash table, a verification information set, and a mapping organization form in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention; Figure 2 In this embodiment, the data verification method for incremental reconstruction of the database may include:

[0079] Step S201 : dividing the to-be-synchronized file and the synchronization reference file into a plurality of to-be-synchronized data blocks and a plurality of synchronization reference data blocks respectively according to the same division method.

[0080] Step S202, set the tuple insertion status flag of the data block to be synchronized to the initial value. The page structure parameters include the tuple insertion status flag (the bit in the T_infomask field that indicates that T_xmin has been submitted), and the tuple insertion status flag is used to indicate whether the last transaction that inserted the tuple has been submitted.

[0081] Step S203, the page header checksum field of the data block to be synchronized is set to an initial value. The page structure parameter also includes a page header checksum field (Pd_checksum), which is used to store the checksum of the data block.

[0082] Step S204: setting the tuple insertion status flag of the synchronization reference data block to an initial value.

[0083] Step S205: setting the page header check field of the synchronization reference data block to an initial value.

[0084] Step S206, calculating a first checksum to be synchronized and a second checksum to be synchronized of each data block to be synchronized.

[0085] Step S207, numbering each data block to be synchronized;

[0086] Step S208: construct a checksum information set with an index subscript according to the serial number of each data block to be synchronized, each first checksum to be synchronized, and each second checksum to be synchronized.

[0087] Step S209: construct a hash table according to each first checksum to be synchronized and each index subscript corresponding thereto, wherein each first checksum to be synchronized is a key value of the hash table, and each index subscript is a hash value of the hash table.

[0088] Step S210, obtaining synchronization reference data blocks one by one.

[0089] Step S211, calculating a first synchronization reference checksum of the acquired synchronization reference data block.

[0090] Step S212, find out whether the calculated first synchronization reference checksum is mapped in the hash table; if not, execute step S213; if yes, if the calculated first synchronization reference checksum is not mapped in the hash table, execute step S215.

[0091] Step S213: determine that there is no verification information in the verification information set that matches the acquired synchronization reference data block.

[0092] Step S214, determine whether the acquired synchronization reference data block belongs to incremental data. After executing this step, the incremental data and the corresponding serial number of the data block to be synchronized are sent from the synchronization reference (server-side) database to the database to be synchronized (client-side). After all the incremental data are sent to the database to be synchronized, the method can be terminated.

[0093] Step S215: using the hash value to retrieve the first to-be-synchronized checksum corresponding to the calculated first synchronization reference checksum from the check information set.

[0094] Step S216, determine whether the calculated first synchronization reference checksum and the retrieved first checksum to be synchronized are the same; if not, execute step S217; if yes, that is, when the calculated first synchronization reference checksum and the retrieved first checksum to be synchronized are the same, execute step S219.

[0095] Step S217: searching the conflict linked list of the hash table for the first checksum to be synchronized that corresponds to the calculated first synchronization reference checksum.

[0096] Step S218, compare the calculated first synchronization reference checksum with the searched first checksum to be synchronized to see if they are the same; if the calculated first synchronization reference checksum is not the same as the searched first checksum to be synchronized, execute step S213; if the calculated first synchronization reference checksum is the same as the searched first checksum to be synchronized, execute step S219.

[0097] Step S219: Calculate the second synchronization reference checksum of the acquired synchronization reference data block.

[0098] Step S220, using the hash value to retrieve the second to-be-synchronized checksum corresponding to the calculated second synchronization reference checksum in the check information set;

[0099] Step S221, determine whether the calculated second synchronization reference checksum is the same as the retrieved second checksum to be synchronized; if not, execute the step of searching the conflict linked list of the hash table for the first checksum to be synchronized corresponding to the calculated first synchronization reference checksum, that is, execute step S217; when the calculated second synchronization reference checksum is the same as the retrieved second checksum to be synchronized, execute step S222.

[0100] Step S222: determine whether there is verification information in the verification information set that matches the acquired synchronization reference data block.

[0101] Step S223: determine whether the acquired synchronization reference data block belongs to matching data. This method can be terminated after all synchronization reference data blocks have been determined.

[0102] Reference Figure 5 It can be understood that the situation that the checksum of the same page (page or data block) may be inconsistent even though the actual stored data content is the same may be caused by the inconsistency of the tuple insertion status identification bit and / or the page header check field of the data block to be synchronized and the synchronization reference data block. Therefore, through steps S202 to S205, the data verification method for incremental reconstruction of the database in this embodiment can limit the misjudgment of incremental data, so as to effectively limit the increase of network bandwidth load caused by misjudged incremental data and improve the synchronization performance of the database.

[0103] Reference Figure 6 and Figure 7 In this embodiment, the data verification method for incremental reconstruction of the database can realize the construction of the verification information set and the hash table through steps S206 to S209, and the synchronization reference data block can be quickly matched with the verification information set through the hash table in steps S207 to S220, thereby ensuring the verification efficiency of the verification process and improving the data synchronization performance. In addition, in steps S207 to S220, two different checksums of the data block to be synchronized and two different checksums of the synchronization reference data block are used for verification, thereby ensuring the accuracy of the verification process and further ensuring the performance of data synchronization.

[0104] Reference Figure 7In this embodiment, if a conflict is generated in the process of constructing the hash table from step S206 to step S209, the data verification method for incremental database reconstruction processes the conflict in the form of a conflict linked list; at the same time, step S214 can further ensure the accuracy of the verification process and the performance of data synchronization. At the same time, it is ensured that the data verification method for incremental database reconstruction can run normally.

[0105] In this embodiment, the step of dividing the to-be-synchronized file into a plurality of to-be-synchronized data blocks, steps S202 to S203, and steps S206 to S207 can be implemented by the to-be-synchronized (client) database. In addition, the step of dividing the synchronization reference file into a plurality of synchronization reference data blocks, steps S204 to S205, and steps S208 to S221 can be implemented by the synchronization reference (server) database. Thus, the verification speed of the data verification method for incremental reconstruction of the database can be guaranteed.

[0106] Figure 3 The present invention is a flowchart of dividing a synchronous reference file into a plurality of synchronous reference data blocks in a data verification method for incremental reconstruction of a database according to an embodiment of the present invention. Figure 3 and Figure 4 In this embodiment, the step of dividing the synchronization reference file into a plurality of synchronization reference data blocks includes:

[0107] Step S301 , dividing a synchronization reference file into a plurality of synchronization data blocks having the same size as the data block to be synchronized.

[0108] Step S302: copy the synchronization data block to obtain the synchronization reference data block.

[0109] It can be understood that the data verification method for incremental reconstruction of the database uses the synchronization reference data block obtained after copying to calculate the first synchronization reference checksum and the second synchronization reference checksum. While realizing data verification, it can avoid data loss and errors in the synchronization reference (server-side) database and ensure data consistency.

[0110] Figure 8 is a schematic diagram of a block structure of a machine-readable storage medium according to an embodiment of the present invention; Figure 8 In this embodiment, a machine executable program 11 may be stored on the machine-readable storage medium 10. When the machine executable program 11 is executed by the processor, the data verification method for incremental reconstruction of the database in the above embodiment is implemented.

[0111] Since the machine-readable storage medium 10 of this embodiment can implement the above-mentioned data verification method for incremental database reconstruction, the machine-readable storage medium 10 of this embodiment also has the beneficial technical effects that can be achieved by the above-mentioned data verification method for incremental database reconstruction.

[0112] Fig. 9 Schematic diagram of a block structure of a computer device according to an embodiment of the present invention. Fig. 9 In this embodiment, the computer device 20 may include a memory 21, a processor 22, and a machine executable program 11 stored in the memory 21 and running on the processor, and the processor 22 implements the data verification method for incremental reconstruction of the database of the above embodiment when executing the machine executable program 11.

[0113] Since the computer device 20 of this embodiment can implement the above-mentioned data verification method for incremental reconstruction of a database, the computer device 20 of this embodiment also has the beneficial technical effects that can be achieved by the above-mentioned data verification method for incremental reconstruction of a database.

[0114] Reference Figure 8 and Fig. 9 It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any machine-readable storage medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or used in combination with these instruction execution systems, devices or equipment.

[0115] For the purpose of the description of the present embodiment, the machine-readable storage medium 10 may be any device that can contain, store, communicate, propagate or transmit a program for use with an instruction execution system, device or apparatus or in conjunction with such instruction execution systems, devices or apparatus. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the machine-readable storage medium 10 may even be paper or other suitable medium on which the program may be printed, because the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting or processing in other suitable ways as necessary, and then stored in a computer memory.

[0116] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0117] The computer device 20 may be, for example, a server, a desktop computer, a notebook computer, a tablet computer, or a smart phone. In some examples, the computer device 20 may be a cloud computing node. The computer device 20 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, a program module may include routines, programs, target programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 20 may be implemented in a distributed cloud computing environment where remote processing devices linked through a communication network perform tasks. In a distributed cloud computing environment, program modules may be located on a local or remote computing system storage medium including a storage device.

[0118] The computer device 20 may include a processor 22 adapted to execute stored instructions, and a memory 21 that provides temporary storage space for the operation of instructions during operation. The processor 22 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 21 may include a random access memory (RAM), a read-only memory, a flash memory, or any other suitable storage system.

[0119] The processor 22 may be connected to an I / O interface (input / output interface) suitable for connecting the computer device 20 to one or more I / O devices (input / output devices) via a system interconnect (e.g., PCI, PCI-Express, etc.). The I / O devices may include, for example, a keyboard and a pointing device, wherein the pointing device may include a touch pad or a touch screen, etc. The I / O devices may be built-in components of the computer device 20, or may be devices externally connected to the computing device.

[0120] Processor 22 can also be linked to a display interface suitable for connecting computer device 20 to a display device through a system interconnect. Display device can include a display screen as a built-in component of computer device 20. Display device can also include a computer monitor, a television or a projector, etc., which are externally connected to computer device 20. In addition, a network interface controller (NIC) can be suitable for connecting computer device 20 to a network through a system interconnect. In some embodiments, NIC can use any suitable interface or protocol (such as Internet Small Computer System Interface, etc.) to transmit data. The network can be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN) or the Internet, etc. Remote devices can be connected to computing devices through a network.

[0121] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A data verification method for incremental reconstruction of a database, comprising: Dividing the to-be-synchronized file and the synchronization reference file into a plurality of to-be-synchronized data blocks and a plurality of synchronization reference data blocks respectively according to the same division method; Unified setting of page structure parameters of the to-be-synchronized data block and the synchronization reference data block; Constructing a verification information set of the data block to be synchronized; Acquire the synchronization reference data blocks one by one, and match the acquired synchronization reference data blocks with the verification information set; If the verification information set does not contain verification information that matches the acquired synchronization reference data block, it is determined that the acquired synchronization reference data block belongs to incremental data.

2. The data verification method for incremental reconstruction of a database according to claim 1, wherein: The page structure parameter includes a tuple insertion status flag, and the tuple insertion status flag is used to identify whether the last transaction that inserted the tuple has been committed; The step of uniformly setting the page structure parameters of the to-be-synchronized data block and the synchronization reference data block comprises: Setting the tuple insertion status flag of the to-be-synchronized data block to an initial value; The tuple insertion status flag of the synchronization reference data block is set to an initial value.

3. The data verification method for incremental reconstruction of a database according to claim 2, wherein: The page structure parameters also include a page header check field, and the page header check field is used to store the checksum of the data block; After the step of setting the tuple insertion status flag of the data block to be synchronized to an initial value, the step of uniformly setting the page structure parameters of the data block to be synchronized further includes: The page header check field of the to-be-synchronized data block is set to an initial value; and, After the step of setting the tuple insertion status flag of the synchronization reference data block to an initial value, the step of uniformly setting the page structure parameters of the synchronization reference data block further includes: The page header check field of the synchronization reference data block is set to an initial value.

4. The data verification method for incremental reconstruction of a database according to claim 3, wherein: The step of constructing the verification information set of the data block to be synchronized comprises: Calculating a first checksum to be synchronized and a second checksum to be synchronized for each of the data blocks to be synchronized; Numbering each of the data blocks to be synchronized; Constructing a check information set with an index subscript according to the number of each of the to-be-synchronized data blocks, each of the first to-be-synchronized checksums, and each of the second to-be-synchronized checksums; A hash table is constructed according to each of the first checksums to be synchronized and each of the index subscripts corresponding thereto, wherein each of the first checksums to be synchronized is a key value of the hash table, and each of the index subscripts is a hash value of the hash table; and, The step of matching the acquired synchronization reference data block with the verification information set comprises: Calculating a first synchronization reference checksum of the acquired synchronization reference data block; Finding whether the calculated first synchronization reference checksum is mapped in the hash table; If not, it is determined that there is no verification information in the verification information set that matches the acquired synchronization reference data block.

5. The data verification method for incremental reconstruction of a database according to claim 4, wherein: In a case where the calculated first synchronization reference checksum does not have a mapping in the hash table, the step of matching the acquired synchronization reference data block with the check information set further includes: Retrieving a first to-be-synchronized checksum corresponding to the calculated first synchronization reference checksum from the check information set using a hash value; Determining whether the calculated first synchronization reference checksum and the retrieved first to-be-synchronized checksum are the same; If not, searching the conflict linked list of the hash table for the first checksum to be synchronized corresponding to the calculated first synchronization reference checksum; Comparing the calculated first synchronization reference checksum and the searched first to-be-synchronized checksum to see if they are the same; In a case where the calculated first synchronization reference checksum is different from the searched first to-be-synchronized checksum, the step of determining that no check information matching the acquired synchronization reference data block exists in the check information set is performed.

6. The data verification method for incremental reconstruction of a database according to claim 5, wherein: In a case where the calculated first synchronization reference checksum and the retrieved first to-be-synchronized checksum are the same, and in a case where the calculated first synchronization reference checksum and the searched first to-be-synchronized checksum are the same, the step of matching the acquired synchronization reference data block with the check information set further includes: Calculating a second synchronization reference checksum of the acquired synchronization reference data block; Retrieving the second to-be-synchronized checksum corresponding to the calculated second synchronization reference checksum from the check information set using the hash value; Determining whether the calculated second synchronization reference checksum is the same as the retrieved second to-be-synchronized checksum; If not, execute the step of searching the conflict linked list of the hash table for the first checksum to be synchronized corresponding to the calculated first synchronization reference checksum.

7. The data verification method for incremental reconstruction of a database according to claim 6, wherein: In a case where the calculated second synchronization reference checksum is the same as the retrieved second to-be-synchronized checksum, the step of matching the acquired synchronization reference data block with the check information set further includes: Determining whether there is verification information in the verification information set that matches the acquired synchronization reference data block; and The data verification method for incremental reconstruction of the database also includes: If the verification information set contains verification information that matches the acquired synchronization reference data block, it is determined that the acquired synchronization reference data block belongs to matching data.

8. The data verification method for incremental reconstruction of a database according to claim 1, wherein: The steps of dividing the synchronization reference file into a plurality of synchronization reference data blocks include: Dividing the synchronization reference file into a plurality of synchronization data blocks having the same size as the data blocks to be synchronized; The synchronization data block is copied to obtain the synchronization reference data block.

9. A machine-readable storage medium having a machine executable program stored thereon, wherein the machine executable program, when executed by a processor, implements the data verification method for incremental reconstruction of a database according to any one of claims 1 to 8.

10. A computer device comprising a memory, a processor, and a machine executable program stored in the memory and running on the processor, wherein the processor implements the data verification method for incremental reconstruction of a database according to any one of claims 1 to 8 when executing the machine executable program.

Images