Database table comparison method and system capable of reducing lock use

Abstract

The invention discloses a database table comparison method and device capable of reducing lock use, and the method comprises the steps: respectively setting local Hash chain tables for leading-out threads of a source end and a target end, and respectively hashing the led-out data into respective local Hash chain tables; respectively setting two global Hash tables corresponding to a source end and a target end for each database table, and when a certain memory block in the local Hash chain table is fully written, inserting the memory block into a chain table corresponding to the global Hash table of the local Hash chain table; when the total memory block number of a certain pair of linked lists of the global Hash table exceeds a limit block, generating a comparison request task; when the comparison thread receives the comparison request task, sub-chain tables corresponding to the Hash values in the comparison request task are extracted from the global Hash tables of the source end and the target end respectively, and deduplication calculation is carried out; and after the whole export task is completed, submitting a final comparison request task to each pair of sub-chain tables of the global Hash table of the target end.

Description

Technical Field

[0001] The present invention relates to the technical field of database synchronization, and in particular to a database table comparison method and system capable of reducing lock usage. Background Art

[0002] In the field of computer software, database software is an indispensable part. In order to meet the needs of database disaster recovery and read-write separation, database synchronization software is usually deployed.

[0003] In the process of synchronizing data with database software, it is inevitable that the source database and the target database may have inconsistent data content or inconsistent data quantity. In order to detect the inconsistency in time, data comparison must be performed.

[0004] The strategy adopted by database synchronization software when comparing tables determines the performance of table comparison. The processing strategy is especially important for large tables. An excellent processing strategy can stably control the amount of calculation and memory usage under the premise of high comparison efficiency.

[0005] In the prior art, database synchronization software generally adopts the following strategies when comparing tables: generally adopt the Hash linked list deduplication strategy, that is, by hashing the md5 value of each data record in the database table, the records are hashed into different linked lists. When one party's record is to be inserted into the Hash table, the linked list is first determined by Hash, and then the corresponding linked list of the other party (target linked list) is checked for duplicate md5. If so, the records of both parties are eliminated. If not, the records are inserted into the linked list of the party.

[0006] With this processing strategy, when each row of records is inserted into the hash table, in order to protect the consistency of the data in the memory and avoid access conflicts, the target linked list needs to be locked, resulting in frequent use of locks and a high probability of lock conflicts, which is particularly obvious when the database table is large. In addition, this practice of allocating memory for a single record seriously affects performance and is prone to memory conflicts. Summary of the invention

[0007] In order to overcome the deficiencies of the above-mentioned prior art, the purpose of the present invention is to provide a database table comparison method and system that can reduce the use of locks, and to improve the speed of database table comparison by optimizing the use of locks by database synchronization software during the database table comparison process.

[0008] To achieve the above object, the present invention provides a database table comparison method that can reduce lock usage, comprising the following steps: Step S1, setting a first local Hash linked list for the source end and the target end export thread respectively, and the export threads of both parties hash the exported data into their respective first local Hash linked lists according to the hash method; Step S2, setting two global hash tables for each database table, corresponding to the source end and the target end respectively; Step S3, taking two sub-linked lists with the same Hash value in the global Hash table of the source end and the target end as a pair of linked lists, when the total number of memory blocks of a pair of sub-linked lists in the global Hash table of the source end and the target end exceeds the limit block, the export thread generates a comparison request task, requesting the comparison thread to compare the corresponding sub-linked lists, and the comparison request task includes the Hash value of the corresponding sub-linked list; Step S4, when receiving the comparison request task, the comparison thread takes out the source end and target end sub-linked lists corresponding to the hash values ​​in the comparison request task from the global hash tables of the source end and the target end respectively, performs a concatenation operation on each data record in the sub-linked list, and performs deduplication calculation; Step S5, after each deduplication calculation is completed, the comparison thread tries again to obtain the sub-linked table with the same hash value as in the S4 task from the global hash table. If new data is obtained, the deduplication calculation is continued. If no new data is obtained, the remaining data is put back into the global hash tables of the source and target ends. Step S6, when the residual data of the source end and the target end in the comparison thread exceeds the set value, write the data to the file corresponding to the hash value in the S4 task; Step S7, when the entire export task is completed, a final comparison request task is submitted for each pair of sub-linked lists of the global Hash table on the source and target ends, and the comparison thread performs a final comparison on the data written to the cache disk file.

[0009] Optionally, in step S1, each export thread hashes the md5 value of the exported data record, and hashes it to the first local Hash linked list of its respective end according to the hash result; when a memory block in a first local Hash linked list is full, the memory block is inserted into the corresponding linked list of the corresponding global Hash table.

[0010] Optionally, the first local hash linked list is composed of several head pointers, each head pointer points to a corresponding memory block block to form a sub-linked list, and in each sub-linked list there is a head pointer pointing to a corresponding memory block. Each memory block has a next pointer and several records, and each record contains the md5 value of the record and a row_ind structure. The row_ind structure locates the specific position of a record, which is reserved for locating the record during comparison.

[0011] Optionally, in step S1, the first 10 bits of the first two bytes of the 16-byte md5 value of the exported data record are used as the conditional hash value to be hashed into the corresponding local hash table.

[0012] Optionally, if the corresponding Hash linked list has not allocated a memory block, a memory block is allocated and the record is stored; if the memory block block has been allocated, the record is directly stored in the corresponding memory block.

[0013] Optionally, in step S2, the head pointer of the global Hash linked list points to the first memory block, and each memory block contains a next pointer and several data records, and the next pointer points to the next memory block.

[0014] Optionally, in step S1, when a certain memory block block in the first local Hash linked list is full, the memory block is inserted into the corresponding linked list of the global Hash table on its own side, and the corresponding global Hash table is locked.

[0015] Optionally, in step S4, when the comparison thread receives the comparison request task, it takes out the source and target terminal linked lists corresponding to the hash values ​​in the comparison request task from the global hash linked lists of the source and target ends and puts them into the comparison local hash table, and concatenates each data record according to the hash value using the row_ind structure.

[0016] Optionally, in step S4, the first 10 bits of the last two bytes of each data record are used as a Hash value, and the data are connected in series using a row_ind structure according to the Hash value.

[0017] To achieve the purpose, the present invention also provides a database table comparison system capable of reducing lock usage, comprising: The export thread includes a source-end export thread and a target-end export thread, which are used to respectively set a first local Hash linked list for the source-end export thread and the target-end export thread, and hash the exported data into the respective first local Hash linked lists according to the hash method. The export thread also sets two global Hash tables for each database table, corresponding to the source end and the target end respectively. In the process of exporting data, when a certain memory block block in the local Hash linked list is full, the memory block is inserted into the corresponding linked list of the global Hash table on its side, and when the total number of memory blocks of a certain pair of linked lists in the global Hash table exceeds the limit block, a comparison request task is generated to request the comparison thread to compare the corresponding linked lists, and when the entire export task is completed, a final comparison request task is submitted for each pair of sub-linked lists of the global Hash tables of the source end and the target end; The comparison thread is used to, when receiving a comparison request task, take out the source and target sub-linked lists corresponding to the hash value in the comparison request task from the global hash tables of the source and target ends respectively, connect each data record in the sub-linked list in series according to the hash value using the row_ind structure, and perform deduplication calculation. When each deduplication calculation is completed, the comparison thread tries again to get the sub-linked list with the same hash value as the comparison request task from the global hash table. If new data is obtained, the deduplication calculation is continued. If there is no new data, the remaining data is put back into the global hash tables of the source and target ends; when the calculated residual memory block exceeds a specific value, the data is written to the disk file corresponding to the hash value.

[0018] Compared with the prior art, the database table comparison method and system of the present invention can reduce the use of locks and significantly reduce the frequency of lock use in the database table comparison process. Locks are only used in the allocation and release of memory blocks during the access operation of the global hash table, and the use of locks is one 256th of the original. The present invention improves the speed of database table comparison by optimizing the use of locks by database synchronization software in the database table comparison process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of the steps of a database table comparison method capable of reducing lock usage according to the present invention; Figure 2 This is a schematic diagram of the memory of the first local Hash linked list in a specific embodiment of the present invention; Figure 3 This is a schematic diagram of the memory of the global Hash linked list in a specific embodiment of the present invention; Figure 4 This is a memory diagram of a comparison local Hash table within a comparison thread in a specific embodiment of the present invention; Figure 5 A system architecture diagram of a database table comparison system capable of reducing lock usage according to the present invention. DETAILED DESCRIPTION

[0020] The following describes the implementation of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0021] Figure 1 The following is a flowchart of the steps of a database table comparison method that can reduce lock usage according to the present invention. Figure 1As shown, a database table comparison method for reducing lock usage of the present invention comprises the following steps: Step S1, setting a first local Hash linked list for the source end and the target end export thread respectively, and the export threads of both parties hash the exported data into their respective first local Hash linked lists according to the Hash method.

[0022] In the present invention, the data export and data comparison of the source and target ends are separated into an export thread and a comparison thread, and then the export threads used for exporting data on both the source and target ends hash the exported data into their respective first local Hash linked lists according to the Hash method. Specifically, the export thread hashes the md5 value of the exported data record, and hashes it into the first local Hash linked lists of the respective ends according to the hash result.

[0023] Specifically, in order to reduce memory allocation conflicts and reduce the use of memory allocation locks, the present invention no longer allocates separate memory for each record, but allocates a memory block for every N (for example, 256) records. The memory block is referred to as a block. Figure 2 As shown, the first local Hash linked list is composed of several head pointers, each of which points to a corresponding memory block to form a sub-linked list, that is, there is a head pointer in each sub-linked list pointing to a corresponding memory block, and there is a next pointer and 0-256 records in each memory block, each record contains the md5 value of the corresponding record and a row_ind structure, wherein the row_ind structure is as follows:

[0024] The row_ind structure locates the specific position of a record and is reserved for locating records during comparison. In the row_ind structure, blk represents a pointer to the memory block of the record, and idx represents the number of the record in the memory block.

[0025] In a specific embodiment of the present invention, a specific approach is as follows: some bits in the 16-byte md5 value of the exported data record are used as the condition of the Hash value to be hashed into the corresponding first local Hash table (that is, the source end and the target end each have their own first local Hash table, and the exported data records are hashed into their own first local Hash tables respectively). The Hash range of this process is called HASH_L1. In some embodiments, the first 10 bits of the first two bytes of the 16-byte md5 value of the exported data record are used as the Hash value by default, that is, the default range is 0-1023. For example, if the first 10 bits of the first two bytes of the md5 value of a data record are 1, the data record should be stored in the sub-linked list corresponding to 1. If no memory block is allocated on the sub-linked list of the corresponding first local Hash linked list, a memory block block is allocated and the data record is stored. If the memory block block has been allocated, the data record is directly stored in the corresponding memory block.

[0026] In the present invention, the first local Hash linked list is only used for internal cache data within the thread, and does not perform matching calculations or locks. Since a separate memory is no longer allocated for each record, but a memory block is allocated for each N (e.g., 256) records, it can be seen that the use of memory allocation locks in the present invention is one-tenth of the original N (e.g., 256).

[0027] Step S2, set two global hash tables for each database table, corresponding to the source end and the target end respectively. When a memory block in the first local hash table is full, the memory block is inserted into the tail of the corresponding sub-list of the global hash table on its own side according to the hash value.

[0028] In a specific embodiment of the present invention, the memory structure of the global Hash linked list is as follows: Figure 3 As shown, the head pointer of the global Hash linked list points to the first memory block, and there is a next pointer and 0-256 records in each memory block. The next pointer points to the next memory block to form a sub-linked list. When a certain memory block block in a certain first local Hash linked list is full, the memory block is inserted into the tail of the corresponding sub-linked list with the same Hash value in the global Hash table of its own side according to the Hash value. For example, when a certain memory block block in the first local Hash linked list of the source end is full, the memory block is inserted into the global Hash table of the source end according to the Hash value. In this process, the corresponding global Hash table needs to be locked to ensure data consistency and avoid access conflicts. It can be seen that the locking frequency of the present invention is one-tenth of the previous N (such as 256).

[0029] Step S3, treats the two sub-linked lists with the same hash value in the global hash table of the source end and the target end as a pair of linked lists. When the total number of memory blocks of a pair of sub-linked lists in the global hash table of the source end and the target end exceeds the limit block, the export thread generates a comparison request task, requesting the comparison thread to compare the corresponding sub-linked lists. The comparison request task contains the hash value of the corresponding sub-linked list.

[0030] Step S4, when the comparison thread receives the comparison request task, it takes out the source and target terminal sub-linked lists corresponding to the hash value in the comparison request task from the global hash linked lists of the source and target ends respectively, connects each data record in the sub-linked list in series according to the hash value using the row_ind structure, and performs deduplication calculation.

[0031] In a specific embodiment of the present invention, two comparison local Hash tables are set up inside the comparison thread, corresponding to the source and target ends respectively, to complete the comparison of specific comparison tasks. When the comparison thread receives a comparison request task, it takes out the source and target terminal linked tables corresponding to the Hash values ​​in the comparison request task from the global Hash linked tables of the source and target ends (and deletes them from the global Hash table at the same time), and concatenates each record according to the different Hash values ​​from the previous one using the row_ind structure, as shown in the following example: Figure 4 As shown, for example, if the Hash value of a row of records in the sub-linked table taken out from the source end is 1, then first determine whether the row_ind structure at the position where the Hash value is 1 in the source end comparison local Hash table is empty. If it is empty, the blk pointer in the structure points to the memory block idx and records the specific position of the record in the memory block. If it is not empty, find the corresponding record according to the row_ind structure, use the row_ind structure in the corresponding record to mark the position of the record, repeat in sequence, connect the records in series, and perform deduplication calculations. Therefore, the present invention no longer involves the problem of memory allocation. The Hash range of the internal comparison local Hash table is HASH_L2, and the default range is 0-1023 (in this embodiment, the first 10 bits of the last two bytes are used as the Hash value by default). The purpose of doing this is that the Hash values ​​of the two sub-linked tables taken out this time are the same. In order to be able to hash into the comparison local Hash table for comparison, a new Hash range is used.

[0032] Step S5, after each deduplication calculation is completed, the comparison thread tries again to retrieve the sub-linked table with the same Hash value as in the S4 task from the global Hash table of the source and target ends. If new data is obtained, the deduplication calculation continues and step S4 is repeated. If there is no new data, the residual data is put back into the global Hash table of the corresponding source and target ends, that is, the residual data is inserted into the tail of the sub-linked table with the same Hash value (according to HASH_L1) in the global Hash table. Since the data of the source and target ends are different, there may be residual data that cannot be deduplicated, and the residual data needs to be put back into the corresponding global Hash table.

[0033] Step S6, when the residual memory blocks calculated by the source and target ends in the comparison thread exceed a specific value (set value), the residual data are written to the file corresponding to the hash value in the S4 task.

[0034] For example, if the number of memory blocks in the comparison thread exceeds 100, the residual data will be written into the file corresponding to the Hash value in the S4 task. That is to say, when the residual memory blocks exceed a specific value, a file corresponding to the Hash value in the S4 task will be created. This is done to prevent too much data in the memory. When the memory occupancy is too large, writing these data into files can reduce the data in the memory, thereby reducing the memory occupancy.

[0035] Step S7, when the entire export task is completed, the export thread submits a final comparison request task for each pair of sub-linked tables in the global hash table of the source and target ends, that is, comparing each pair of sub-linked tables in the memory with the data written to the cache disk file. The data in the memory is compared first, and then the data in the disk file is compared.

[0036] That is to say, when the entire export task is completed, a final comparison task is submitted for each pair of linked lists of the source and target of the global Hash table. After the comparison thread has compared the memory data, it needs to do the final finishing work. That is, start comparing the cached disk files. When the disk file is taken out to the comparison thread, due to the large amount of data, the Hash range is expanded to shorten the length of the sub-linked list in the local Hash table of the comparison thread. At this time, the HASH range used is HASH_L3, and the default range is 0-512K (the first 19 bits of the first three bytes are used as the Hash value by default).

[0037] Figure 5 This is a system architecture diagram of a database table comparison system that can reduce lock usage in the present invention. Figure 5 As shown, the present invention provides a database table comparison system capable of reducing lock usage, comprising: The export thread 51 includes a source-end export thread and a target-end export thread, which are used to set local Hash linked lists for the source-end export thread and the target-end export thread respectively, and hash the exported data into their respective local Hash linked lists according to the Hash method. The export thread 51 also sets two global Hash tables for each database table, corresponding to the source end and the target end respectively. During the data export process, when a memory block block in the local Hash linked list is full, the memory block is inserted into the corresponding linked list of the global Hash table on its own side, and when the total number of memory blocks of a pair of linked lists in the global Hash table exceeds the limit block, a comparison request task is generated, requesting the comparison thread to compare the corresponding linked lists, and when the entire export task is completed, a final comparison request task is submitted for each pair of sub-linked lists of the global Hash table of the source end and the target end, that is, the data written into the cache disk file is finally compared.

[0038] In the present invention, the data export and data comparison of the source and target ends are separated into an export thread and a comparison thread, and then the export threads of the source and target ends respectively hash the exported data into their respective local Hash linked lists according to the Hash method. Specifically, the source / target end export thread hashes the md5 of the exported data record, and hashes it into the local Hash linked lists of the respective ends according to the hash result.

[0039] Specifically, in order to reduce memory allocation conflicts and reduce the use of memory allocation locks, the present invention no longer allocates separate memory for each record, but allocates a memory block for every N (for example, 256) records. Here, the memory block is called a block. The local Hash linked list is composed of a number of head pointers, each of which points to a corresponding memory block block to form a sub-linked list, that is, there is a head pointer in each sub-linked list pointing to a corresponding memory block block, and there is a next pointer and 0-256 records in each memory block. Each record contains the md5 value of the record and the row_ind structure. The row_ind structure locates the specific position of a record and is reserved for locating records during comparison. In the row_ind structure, blk represents a pointer to the memory block of the record, and idx represents the number of the record in the memory block. In a specific embodiment of the present invention, the specific method is: some bits in the 16-byte md5 value of the exported data record are used as the conditional hash of the Hash value to the corresponding local Hash table. The Hash range of this process is called HASH_L1, and the default range is 0-1023 (the first 10 bits of the first two bytes of the 16-byte md5 value of the exported data record are used as the Hash value by default). If the corresponding Hash linked list has not allocated a memory block, a memory block block is allocated and the record is stored. If the memory block block has been allocated, the record is directly stored in the corresponding memory block.

[0040] In the present invention, the local Hash linked list is only used for internal cache data within the thread, and no matching calculation is performed, and no locking is required. Since a separate memory is no longer allocated for each record, but a memory block is allocated for each N (e.g., 256) records, it can be seen that the use of memory allocation locks in the present invention is one-tenth of the original N (e.g., 256).

[0041] In a specific embodiment of the present invention, the memory structure of the global Hash linked list is as follows: Figure 3 As shown, the head pointer of the global Hash linked list points to the first memory block, and there is a next pointer and 0-256 records in each memory block. The next pointer points to the next memory block, and each memory block forms a sub-linked list. When a certain memory block block in the local Hash linked list is full, the corresponding global Hash table needs to be locked in the process of inserting the memory block into the corresponding linked list of the global Hash table on its own side, so as to ensure data consistency and avoid access conflicts. It can be seen that the locking frequency of the present invention is one-tenth of the previous N (such as 256).

[0042] When the export thread finds that the total number of memory blocks of a pair of linked lists in the global hash table exceeds the limit, it generates a comparison request task and requests the comparison thread to compare the corresponding linked lists. The comparison thread 52 is used to, when receiving a comparison request task, take out the source and target sub-linked lists corresponding to the hash value in the comparison request task from the global hash linked lists of the source and target ends respectively, connect each data record in the sub-linked list in series using the row_ind structure according to the hash value, and perform deduplication calculation. When each deduplication calculation is completed, the comparison thread 52 will try again to obtain the sub-linked list with the same hash value as the comparison request task. If new data is obtained, the deduplication calculation will continue. If there is no new data, the remaining data will be put back into the global hash table of the source and target ends; when the calculated residual memory block exceeds a specific value, the data will be written to the disk file corresponding to the hash value.

[0043] In a specific embodiment of the present invention, a comparison local Hash table is set inside the comparison thread 52 to complete the comparison of a specific comparison task. When the comparison thread 52 receives the comparison request task from the export thread, it takes out the source and target terminal linked lists corresponding to the specific Hash value from the global Hash linked lists of the source and target ends and puts them into the comparison local Hash table, and concatenates each record according to the Hash value using the row_ind structure, such as Figure 4 As shown, deduplication calculation is performed, so the present invention no longer involves memory allocation issues. The Hash range of the internal comparison local Hash table is HASH_L2, and the default range is 0-1023 (in this embodiment, the first 10 bits of the last two bytes are used as the Hash value by default).

[0044] After each deduplication calculation is completed, the comparison thread tries again to obtain the data of the specific linked list from the global hash table. If new data is obtained, the deduplication calculation continues. If there is no new data, the remaining data is put back into the global hash table.

[0045] When the calculated residual memory block exceeds a specific value, the data is written to the file corresponding to the hash value.

[0046] When the final comparison request task is received, the comparison thread performs a final comparison on the data written to the cache disk file.

[0047] It can be seen that the present invention significantly reduces the frequency of lock usage during database table comparison. Locks are only used in the allocation and release of memory blocks during access operations in the global hash table, and the amount of lock usage is one 256th of the original.

[0048] The above embodiments are only illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person skilled in the art may modify and change the above embodiments without violating the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be as set forth in the claims.

Claims

1. A database table comparison method that can reduce lock usage, comprising the following steps: Step S1, setting a first local Hash linked list for the source end and the target end export thread respectively, and the export threads of both parties hash the exported data into their respective first local Hash linked lists according to the hash method; Step S2, setting two global hash tables for each database table, corresponding to the source end and the target end respectively; Step S3, taking two sub-linked lists with the same Hash value in the global Hash table of the source end and the target end as a pair of linked lists, when the total number of memory blocks of a pair of sub-linked lists in the global Hash table of the source end and the target end exceeds the limit block, the export thread generates a comparison request task, requesting the comparison thread to compare the corresponding sub-linked lists, and the comparison request task includes the Hash value of the corresponding sub-linked list; Step S4, when receiving the comparison request task, the comparison thread takes out the source end and target end sub-linked lists corresponding to the hash values ​​in the comparison request task from the global hash tables of the source end and the target end respectively, performs a concatenation operation on each data record in the sub-linked list, and performs deduplication calculation; Step S5, after each deduplication calculation is completed, the comparison thread tries again to obtain the sub-linked table with the same hash value as in the S4 task from the global hash table. If new data is obtained, the deduplication calculation is continued. If no new data is obtained, the remaining data is put back into the global hash tables of the source and target ends. Step S6, when the residual data of the source end and the target end in the comparison thread exceeds the set value, write the data to the file corresponding to the hash value in the S4 task; Step S7, when the entire export task is completed, a final comparison request task is submitted for each pair of sub-linked lists of the global Hash table on the source and target ends, and the comparison thread performs a final comparison on the data written to the cache disk file.

2. A database table comparison method capable of reducing lock usage as claimed in claim 1, characterized in that: In step S1, each export thread hashes the md5 value of the exported data record, and hashes it to the first local Hash linked list of each end according to the hash result; when a memory block in a first local Hash linked list is full, the memory block is inserted into the corresponding linked list of the corresponding global Hash table.

3. A database table comparison method capable of reducing lock usage as claimed in claim 2, characterized in that: The first local Hash linked list is composed of several head pointers, each of which points to the corresponding memory block block to form a sub-linked list. In each sub-linked list, there is a head pointer pointing to the corresponding memory block. In each memory block, there is a next pointer and several records. Each record contains the md5 value of the record and a row_ind structure. The row_ind structure locates the specific position of a record, which is reserved for locating the record during comparison.

4. A database table comparison method capable of reducing lock usage as claimed in claim 3, characterized in that: In step S1, the first 10 bits of the first two bytes of the 16-byte md5 value of the derived data record are used as the conditional hash value to be hashed into the corresponding local hash table.

5. A database table comparison method capable of reducing lock usage as claimed in claim 4, characterized in that: If the corresponding Hash linked list has not allocated a memory block, a memory block is allocated and the record is stored. If a memory block has been allocated, the record is directly stored in the corresponding memory block.

6. A database table comparison method capable of reducing lock usage as claimed in claim 4, characterized in that: In step S2, the head pointer of the global Hash linked list points to the first memory block, and each memory block contains a next pointer and several data records, and the next pointer points to the next memory block.

7. A database table comparison method capable of reducing lock usage as claimed in claim 6, characterized in that: In step S1, when a certain memory block block in the first local Hash linked list is full, the memory block is inserted into the corresponding linked list of the global Hash table on its own side, and the corresponding global Hash table is locked.

8. A database table comparison method capable of reducing lock usage as claimed in claim 7, characterized in that: In step S4, when the comparison thread receives the comparison request task, it takes out the source and target terminal linked lists corresponding to the hash values ​​in the comparison request task from the global hash linked lists of the source and target ends and puts them into the comparison local hash table, and concatenates each data record according to the hash value using the row_ind structure.

9. A database table comparison method capable of reducing lock usage as claimed in claim 8, characterized in that: In step S4, the first 10 bits of the last two bytes of each data record are used as the Hash value, and the row_ind structure is used to connect them in series according to the Hash value.

10. A database table comparison system capable of reducing lock usage, comprising: The export thread includes a source-end export thread and a target-end export thread, which are used to respectively set a first local Hash linked list for the source-end export thread and the target-end export thread, and hash the exported data into the respective first local Hash linked lists according to the hash method. The export thread also sets two global Hash tables for each database table, corresponding to the source end and the target end respectively. In the process of exporting data, when a certain memory block block in the local Hash linked list is full, the memory block is inserted into the corresponding linked list of the global Hash table on its side, and when the total number of memory blocks of a certain pair of linked lists in the global Hash table exceeds the limit block, a comparison request task is generated to request the comparison thread to compare the corresponding linked lists, and when the entire export task is completed, a final comparison request task is submitted for each pair of sub-linked lists of the global Hash tables of the source end and the target end; The comparison thread is used to, when receiving a comparison request task, take out the source and target sub-linked lists corresponding to the hash value in the comparison request task from the global hash tables of the source and target ends respectively, connect each data record in the sub-linked list in series according to the hash value using the row_ind structure, and perform deduplication calculation. When each deduplication calculation is completed, the comparison thread tries again to get the sub-linked list with the same hash value as the comparison request task from the global hash table. If new data is obtained, the deduplication calculation is continued. If there is no new data, the remaining data is put back into the global hash tables of the source and target ends; when the calculated residual memory block exceeds a specific value, the data is written to the disk file corresponding to the hash value.

Images