A database reading acceleration method, device, equipment and storage medium

Leveraging the parallel computing and storage advantages of FPGAs, and utilizing Bloom filters and search bitmaps, the low read performance of LSM trees and the false positive problem of Bloom filters are solved, thereby improving database read efficiency and reducing the probability of false positives.

CN116383244BActive Publication Date: 2026-03-03山东浪潮数据库技术有限公司
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-28
Publication Date
2026-03-03

AI Technical Summary

Technical Problem

LSM trees sacrifice read performance to improve write performance, while Bloom filters have the potential for false positive matches, resulting in low database read efficiency.

Method used

Leveraging the parallel computing advantages of FPGA, a Bloom filter is used to perform parallel hash calculations on key-value information. The search bitmap in the log structure merge tree is traversed to determine whether there are zero values ​​in the bit information. The sorted string table identifier corresponding to the target search bitmap is then output for database reading.

Benefits of technology

It improves database read efficiency, reduces the number of memory accesses to disk, and lowers the false positive probability of the Bloom filter.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116383244B_ABST
    Figure CN116383244B_ABST
Patent Text Reader

Abstract

The application discloses a database reading acceleration method and device, equipment and a storage medium, and relates to the field of database acceleration. The method comprises the following steps: obtaining corresponding address information by performing hash value parallel calculation on the obtained key value information by using a Bloom filter; traversing search bitmaps corresponding to each sorting string table in an LSM tree to obtain bit information corresponding to the address information from each search bitmap; determining whether each bit information has a zero value, and if not, taking the search bitmap corresponding to the bit information without the zero value as a target search bitmap, and outputting a target sorting string table identifier corresponding to the target search bitmap to a memory, so that the memory performs a reading operation on the database. The application is applied to an FPGA, the parallel computing advantage of the FPGA is exerted, the search bitmap is traversed by using the FPGA, the reading performance on the database is improved, and the storage advantage of the FPGA can reduce the false positive probability of the Bloom filter.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of database acceleration, and in particular to a method, apparatus, device, and storage medium for accelerating database reading. Background Technology

[0002] LSM (Log-Structured Merge-Tree) sacrifices some read performance to significantly improve write performance. The more SSTable (Sorted String Table) files there are, the lower the read performance of the LSM tree becomes. Some LSM tree implementations use Bloom filters to optimize read performance.

[0003] A Bloom filter is a space-efficient probabilistic algorithm used to test whether an element exists in a set. Its drawback is the possibility of false positives (matches that are actually false but mistakenly identified as true), but it does not have the possibility of false negatives. In other words, a query may return a result that is either in the set or definitely not in the set. Since an empty Bloom filter is an m-bit array or bitmap, and k hash functions need to be defined, each mapping an element to one bit in the bitmap, the probability of a false positive depends on the values ​​of m and k. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a database read acceleration method, apparatus, device, and storage medium, which, when applied to an FPGA, leverages the parallel computing advantages of the FPGA and utilizes the FPGA to traverse the search bitmap, thereby improving database read performance. Simultaneously, the storage advantages of the FPGA can reduce the false positive probability of the Bloom filter. The specific solution is as follows:

[0005] Firstly, this application provides a database read acceleration method applied to FPGA, including:

[0006] The key-value information is obtained through a preset reading interface, and the hash value of the key-value information is calculated in parallel using a Bloom filter to obtain the corresponding address information.

[0007] Traverse the search bitmaps corresponding to each sorted string table in the log structure merge tree, and obtain the bit information corresponding to the address information from each of the search bitmaps;

[0008] Each bit information is determined to have a zero value. If no zero value is found, the search bitmap corresponding to the bit information without a zero value is used as the target search bitmap, and the target sorting string table identifier corresponding to the target search bitmap is output to memory, so that the memory can perform a read operation on the database based on the target sorting string table identifier.

[0009] Optionally, the step of using a Bloom filter to perform parallel hash calculations on the key-value information to obtain the corresponding address information includes:

[0010] The key-value information is input into a Bloom filter, and a hash function is used to calculate the key-value information to obtain the corresponding summary information.

[0011] The summary information is equally divided into a first preset number of data parts, and the first preset number of hash functions are used to perform parallel calculations on the corresponding first preset number of data parts to obtain the first preset number of hash values.

[0012] The first preset number of hash values ​​are concatenated to obtain a concatenated hash value, and the concatenated hash value is divided according to a preset splitting rule to obtain a split hash value.

[0013] A second preset number of segmented data are selected from the segmented hash values ​​using a preset selection rule to obtain the corresponding second preset number of address information.

[0014] Optionally, the step of selecting a second preset number of segmented data from the segmented hash values ​​using a preset selection rule to obtain the corresponding second preset number of address information includes:

[0015] A second preset number of segmented data are selected from the segmented hash value using a preset selection rule, and it is determined whether each segmented data is greater than the number of bits corresponding to the search bitmap.

[0016] If it is greater than, then the corresponding segmented data is determined, and the segmented data is used to perform a modulo operation on the number of bits corresponding to the search bitmap to obtain the corresponding address information;

[0017] If it is less than or equal to, then the corresponding segmented data is determined, and the segmented data is identified as the corresponding address information.

[0018] Optionally, the preset read interface is a data transmission interface implemented based on a PCIe bus driver.

[0019] Optionally, before the search bitmap corresponding to each sorted string table in the traversal log structure merge tree, the method further includes:

[0020] When writing the immutable table in memory from the log structure merge tree to the sorted string table at the first level on disk, a Bloom filter is used to process the initial key values ​​contained in the immutable table to generate a search bitmap corresponding to the sorted string table.

[0021] Optionally, determining whether a zero value exists in each of the bit information includes:

[0022] Perform a logical AND operation on each of the stated bit information to obtain the corresponding logical AND results;

[0023] The presence of a zero value in each bit information is determined based on whether the result of each logical AND operation is zero.

[0024] Optionally, after determining whether a zero value exists in each of the bit information, the method further includes:

[0025] If each bit contains a zero value, an invalid identifier is output indicating that the key value information does not exist in any of the search bitmaps.

[0026] Secondly, this application provides a database read acceleration device applied to an FPGA, comprising:

[0027] The hash value calculation module is used to obtain key value information through a preset reading interface, and to perform parallel hash value calculation on the key value information using a Bloom filter to obtain the corresponding address information;

[0028] The information acquisition module is used to traverse the search bitmaps corresponding to each sorted string table in the log structure merging tree, and to acquire the bit information corresponding to the address information from each of the search bitmaps.

[0029] The database reading module is used to determine whether there is a zero value in each bit information. If there is no zero value, the search bitmap corresponding to the bit information without a zero value is used as the target search bitmap, and the target sorting string table identifier corresponding to the target search bitmap is output to the memory, so that the memory can perform a read operation on the database based on the target sorting string table identifier.

[0030] Thirdly, this application provides an electronic device, comprising:

[0031] Memory, used to store computer programs;

[0032] A processor for executing the computer program to implement the aforementioned database read acceleration method.

[0033] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned database read acceleration method.

[0034] In this application, key-value information is obtained through a preset reading interface, and a Bloom filter is used to perform parallel hash calculations on the key-value information to obtain the corresponding address information. The search bitmaps corresponding to each sorted string table in the log structure merge tree are traversed, and bit information corresponding to the address information is obtained from each search bitmap. It is determined whether each bit information contains a zero value. If not, the search bitmap corresponding to the bit information without a zero value is used as the target search bitmap, and the target sorted string table identifier corresponding to the target search bitmap is output to memory, so that the memory can perform read operations on the database based on the target sorted string table identifier. Therefore, this application, by applying it to an FPGA, fully leverages the parallel computing advantages of the FPGA and uses the FPGA to traverse the search bitmaps corresponding to each sorted string table to determine whether key-value information exists in a certain sorted string table, reducing the number of memory accesses to the disk and thus improving the database read efficiency. Furthermore, this application utilizes the storage advantages of the FPGA to increase the values ​​of the Bloom filter's search bitmap, thereby reducing the false positive probability of the Bloom filter. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1 This is a flowchart of a database reading acceleration method disclosed in this application;

[0037] Figure 2 This is a schematic diagram of an FPGA structure disclosed in this application;

[0038] Figure 3 This is a schematic diagram illustrating one method of determining bit information disclosed in this application;

[0039] Figure 4 This is a schematic diagram of the structure of a database reading acceleration device disclosed in this application;

[0040] Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] LSM trees sacrifice some read performance to significantly improve write performance. The more SSTable files there are, the lower the read performance of the LSM tree becomes. Some LSM tree implementations use Bloom filters to optimize read performance, but Bloom filters are prone to false positives. Therefore, this application provides a database read acceleration method. By applying it to an FPGA, it leverages the parallel computing advantages of the FPGA and uses the FPGA to traverse the search bitmap, improving database read performance. Simultaneously, the storage advantages of the FPGA can reduce the false positive probability of the Bloom filter.

[0043] See Figure 1 As shown, this embodiment of the invention discloses a database read acceleration method applied to FPGA, including:

[0044] Step S11: Obtain key-value information through a preset reading interface, and use a Bloom filter to perform parallel hash value calculation on the key-value information to obtain the corresponding address information.

[0045] In this embodiment, as Figure 2 As shown, the preset read interface is a data transmission interface implemented based on the PCIe bus (Peripheral Component Interconnect express, high-speed serial computer expansion bus standard). The FPGA obtains the key information through the PCIe bus and performs parallel hash calculation on the obtained key information through a Bloom filter to obtain the address information.

[0046] In this embodiment, the above-mentioned parallel hash calculation of key-value information using a Bloom filter to obtain corresponding address information may include: inputting the key-value information into a Bloom filter and calculating the key-value information using a hash function to obtain corresponding digest information; equally dividing the digest information into a first preset number of data parts, and using the first preset number of hash functions to perform parallel calculations on the corresponding first preset number of data parts to obtain the first preset number of hash values; concatenating the first preset number of hash values ​​to obtain a concatenated hash value, and dividing the concatenated hash value according to a preset segmentation rule to obtain a segmented hash value; and selecting a second preset number of segmented data from the segmented hash values ​​using a preset selection rule to obtain the corresponding second preset number of address information. It can be understood that, as... Figure 3 As shown, the key-value information (key) is input into a Bloom filter, and a highly secure hash function (k) is used to calculate the hash value of the obtained key-value information to obtain the digest information. The digest information is then divided into n equal parts, namely s1, s2, ..., s... n and using n hash functions (k1, ..., k n The data is processed in parallel to obtain n hash values. These n hash values ​​are then concatenated and further divided according to a preset splitting rule. Ten of these splits, n1, n2, ..., n, are then selected. 10 This process yields 10 corresponding address information. The preset segmentation rule can be an equal segmentation rule or other segmentation rules. It should be noted that the hash functions used in this application include, but are not limited to, SHA256 and SHA512, and either completely different hash functions can be selected, or one or two hash functions can be reused.

[0047] In this embodiment, the specific steps for determining the second preset quantity are as follows: Due to the false positive probability of the Bloom filter... Where n represents the number of elements already added, k represents the number of hash rounds performed after inserting an element, m represents the size of the bit array of the Bloom filter, and P represents the false positive probability when n elements are inserted. When m is 20 times n, that is, when k is 10, the false positive probability of the Bloom filter is the smallest. Furthermore, when m is 20 times n, that is, when k is 10, the false positive probability of the Bloom filter is the smallest. Therefore, this application sets the second preset quantity to 10.

[0048] In this embodiment, the above-mentioned selection of a second preset number of segmented data from the segmented hash value using a preset selection rule to obtain the corresponding second preset number of address information may include selecting a second preset number of segmented data from the segmented hash value using a preset selection rule, and determining whether each segmented data is greater than the number of bits corresponding to the search bitmap; if it is greater, the corresponding segmented data is determined, and the segmented data is used to perform a modulo operation on the number of bits corresponding to the search bitmap to obtain the corresponding address information; if it is less than or equal to, the corresponding segmented data is determined, and the segmented data is determined as the corresponding address information. It can be understood that, as Figure 3 As shown, when selecting 10 data points, namely n1, n2, ..., n... 10 Next, it is necessary to further determine each segmented data (n1, n2, ..., n) 10 If the value of the segmented data is greater than the number of bits corresponding to the search bitmap, then a remainder operation needs to be performed on the number of bits corresponding to the search bitmap using this segmented data, and the remainder is used to determine the address information corresponding to this segmented data. If the value is less than or equal to this, then this segmented data can be directly determined as the corresponding address information, thus obtaining the address information corresponding to the 10 segmented data (n1, n2, ..., n). 10 The 10 address information corresponding to (h1, h2, ..., h) 10 For example, if the segmented data n1 is 4, which is less than the number of bits corresponding to the search bitmap, then 4 is directly used as the corresponding address information.

[0049] Step S12: Traverse the search bitmaps corresponding to each sorted string table in the log structure merge tree, and obtain the bit information corresponding to the address information from each search bitmap.

[0050] In this embodiment, before traversing the search bitmaps corresponding to each sorted string table in the log structure merge tree, it may further include processing the initial key values ​​contained in the immutable table located in memory to generate a search bitmap corresponding to the sorted string table when writing the immutable table in memory to the sorted string table located in the first layer on disk using a Bloom filter. It can be understood that when writing the immutable table to the sorted string table (SSTable) in the L0 layer on disk, the FPGA obtains the initial key values ​​contained in the immutable table through the PCIe bus, and performs parallel hash calculations on the initial key values ​​using a Bloom filter to obtain the corresponding address information. The bits corresponding to the address information in the initial search bitmap are set to 1 to obtain the search bitmap corresponding to each sorted string table. In the initial search bitmap, all bits are 0. Further, as... Figure 2 and Figure 3As shown, the FPGA traverses the search bitmaps corresponding to each sorted string table in the log structure merge tree to obtain the address information (h1, h2, ..., h...). 10 The corresponding bit information.

[0051] Step S13: Determine whether there is a zero value in each bit information. If not, take the search bitmap corresponding to the bit information without a zero value as the target search bitmap, and output the target sorting string table identifier corresponding to the target search bitmap to memory, so that the memory can perform a read operation on the database based on the target sorting string table identifier.

[0052] In this embodiment, determining whether a zero value exists in each bit information can include performing a logical AND operation on each bit information to obtain corresponding logical AND results; determining whether a zero value exists in each bit information based on whether the logical AND results are zero. It can be understood that for any bit information, a logical AND operation is performed on that bit information to obtain a corresponding logical AND result (0 or 1). If the logical AND result is 0, it indicates that a zero value exists in that bit information, meaning there is no corresponding key-value information in the sorted string table; if the logical AND result is 1, it indicates that all bits in that bit information are 1, meaning there may be corresponding key-value information in the sorted string table, and then the target sorted string table identifier corresponding to the search bitmap is output to memory. Further, as... Figure 3 As shown, the search bitmap corresponding to the bits containing no zero values ​​is determined as the target search bitmap. Then, the target sorted string table identifier corresponding to the target search bitmap is output to the DMA (Direct Memory Access) engine via DDR (Double Data Rate) so that memory can perform corresponding read operations on the database using the target sorted string table identifier. If all bit information contains zero values, it indicates that no corresponding key-value information exists in any of the sorted string tables. An invalid identifier indicating that no key-value information exists in any search bitmap is then output to prompt the user to re-enter the key-value information.

[0053] Therefore, this application, by applying it to FPGA, fully leverages the parallel computing advantages of FPGA and uses FPGA to traverse the search bitmaps corresponding to each sorted string table to determine whether key value information exists in a certain sorted string table, thereby reducing the number of memory accesses to disk and improving the database reading efficiency. In addition, this application utilizes the storage advantages of FPGA to increase the values ​​of the Bloom filter's search bitmap, thereby reducing the false positive probability of the Bloom filter.

[0054] See Figure 4 As shown, this embodiment of the invention discloses a database read acceleration device applied to an FPGA, comprising:

[0055] The hash value calculation module 11 is used to obtain key value information through a preset reading interface and perform parallel hash value calculation on the key value information using a Bloom filter to obtain the corresponding address information;

[0056] Information acquisition module 12 is used to traverse the search bitmaps corresponding to each sorted string table in the log structure merging tree, and to acquire bit information corresponding to the address information from each search bitmap.

[0057] The database reading module 13 is used to determine whether there is a zero value in each of the bit information. If there is no zero value, the search bitmap corresponding to the bit information without a zero value is used as the target search bitmap, and the target sorting string table identifier corresponding to the target search bitmap is output to the memory, so that the memory can perform a read operation on the database based on the target sorting string table identifier.

[0058] Therefore, this application, by applying it to FPGA, fully leverages the parallel computing advantages of FPGA and uses FPGA to traverse the search bitmaps corresponding to each sorted string table to determine whether key value information exists in a certain sorted string table, thereby reducing the number of memory accesses to disk and improving the database reading efficiency. In addition, this application utilizes the storage advantages of FPGA to increase the values ​​of the Bloom filter's search bitmap, thereby reducing the false positive probability of the Bloom filter.

[0059] In some specific embodiments, the hash value calculation module 11 may specifically include:

[0060] The summary information determination unit is used to input the key-value information into a Bloom filter and use a hash function to calculate the key-value information to obtain the corresponding summary information;

[0061] The hash value calculation unit is used to equally divide the digest information into a first preset number of data parts, and use the first preset number of hash functions to perform parallel calculations on the corresponding first preset number of data parts to obtain the first preset number of hash values.

[0062] The hash value concatenation unit is used to concatenate the first preset number of hash values ​​to obtain a concatenated hash value, and to divide the concatenated hash value according to a preset splitting rule to obtain a split hash value.

[0063] The address information determination submodule is used to select a second preset number of segmented data from the segmented hash values ​​using preset selection rules, so as to obtain the corresponding second preset number of address information.

[0064] In some specific embodiments, the address information determination submodule may specifically include:

[0065] The bit depth determination unit is used to select a second preset number of segmented data from the segmented hash value using a preset selection rule, and to determine whether each segmented data is greater than the bit depth corresponding to the search bitmap.

[0066] The data modulo unit is used to determine the corresponding segmented data if it is greater than the specified value, and to perform a modulo operation on the number of bits corresponding to the search bitmap using the segmented data to obtain the corresponding address information.

[0067] The data determination unit is used to determine the corresponding segmented data if it is less than or equal to the address information.

[0068] In some specific embodiments, the database read acceleration device may further include:

[0069] The search bitmap generation unit is used to process the initial key values ​​contained in the immutable table in the log structure merge tree located in memory into the sorted string table located in the first level on the disk using a Bloom filter, so as to generate a search bitmap corresponding to the sorted string table.

[0070] In some specific embodiments, the database reading module 13 may specifically include:

[0071] A bit-level logical AND unit is used to perform logical AND operations on each bit of information to obtain the corresponding logical AND results;

[0072] A zero-value determination unit is used to determine whether a zero value exists in each bit information based on whether the result of each logical AND operation is zero.

[0073] In some specific embodiments, the database read acceleration device may further include:

[0074] An invalid flag output unit is used to output an invalid flag indicating that the key value information does not exist in any of the search bitmaps if all of the bit information contains a zero value.

[0075] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0076] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the database read acceleration method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0077] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0078] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0079] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the database read acceleration method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0080] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned database read acceleration method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0082] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0083] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0084] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0085] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for accelerating database read operations, characterized in that, Applied to FPGAs, including: The key-value information is obtained through a preset reading interface, and the hash value of the key-value information is calculated in parallel using a Bloom filter to obtain the corresponding address information. Traverse the search bitmaps corresponding to each sorted string table in the log structure merge tree, and obtain the bit information corresponding to the address information from each of the search bitmaps; Determine whether there is a zero value in each bit information. If there is no zero value, take the search bitmap corresponding to the bit information without zero value as the target search bitmap, and output the target sorting string table identifier corresponding to the target search bitmap to memory, so that the memory can perform a read operation on the database based on the target sorting string table identifier. The step of using a Bloom filter to perform parallel hash calculations on the key-value information to obtain the corresponding address information includes: The key-value information is input into a Bloom filter, and a hash function is used to calculate the key-value information to obtain the corresponding summary information. The summary information is equally divided into a first preset number of data parts, and the first preset number of hash functions are used to perform parallel calculations on the corresponding first preset number of data parts to obtain the first preset number of hash values. The first preset number of hash values ​​are concatenated to obtain a concatenated hash value, and the concatenated hash value is divided according to a preset splitting rule to obtain a split hash value. A second preset number of segmented data are selected from the segmented hash values ​​using a preset selection rule to obtain the corresponding second preset number of address information; The step of selecting a second preset number of segmented data from the segmented hash values ​​using a preset selection rule to obtain the corresponding second preset number of address information includes: A second preset number of segmented data are selected from the segmented hash value using a preset selection rule, and it is determined whether each segmented data is greater than the number of bits corresponding to the search bitmap. If it is greater than, then the corresponding segmented data is determined, and the segmented data is used to perform a modulo operation on the number of bits corresponding to the search bitmap to obtain the corresponding address information; If it is less than or equal to, then the corresponding segmented data is determined, and the segmented data is identified as the corresponding address information.

2. The database read acceleration method according to claim 1, characterized in that, The preset read interface is a data transmission interface implemented based on the PCIe bus driver.

3. The database read acceleration method according to claim 1, characterized in that, Before the search bitmap corresponding to each sorted string table in the traversal log structure merge tree, the following is also included: When writing the immutable table in memory from the log structure merge tree to the sorted string table at the first level on disk, a Bloom filter is used to process the initial key values ​​contained in the immutable table to generate a search bitmap corresponding to the sorted string table.

4. The database read acceleration method according to claim 1, characterized in that, The step of determining whether each bit information contains a zero value includes: Perform a logical AND operation on each of the stated bit information to obtain the corresponding logical AND results; The presence of a zero value in each bit information is determined based on whether the result of each logical AND operation is zero.

5. The database read acceleration method according to any one of claims 1 to 4, characterized in that, After determining whether each bit information contains a zero value, the method further includes: If each bit contains a zero value, an invalid identifier is output indicating that the key value information does not exist in any of the search bitmaps.

6. A database read acceleration device, characterized in that, Applied to FPGA, the database read acceleration device is used to implement the database read acceleration method as described in any one of claims 1 to 5, the database read acceleration device comprising: The hash value calculation module is used to obtain key value information through a preset reading interface, and to perform parallel hash value calculation on the key value information using a Bloom filter to obtain the corresponding address information; The information acquisition module is used to traverse the search bitmaps corresponding to each sorted string table in the log structure merging tree, and to acquire the bit information corresponding to the address information from each of the search bitmaps. The database reading module is used to determine whether there is a zero value in each bit information. If there is no zero value, the search bitmap corresponding to the bit information without a zero value is used as the target search bitmap, and the target sorting string table identifier corresponding to the target search bitmap is output to the memory, so that the memory can perform a read operation on the database based on the target sorting string table identifier.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the database read acceleration method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store computer programs, which, when executed by a processor, implement the database read acceleration method as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • =Data processing method in bloom filter and bloom filter

    CN109145158A

  • Base number statistics method, device, system and equipment and computer readable storage medium

    CN111563109A