Self-adaptive range query filtering method, system and equipment and storage medium

Abstract

The invention discloses a self-adaptive range query filtering method, system and device and a storage medium. The method comprises the steps that each key in an input data set is segmented into a prefix and a suffix, the keys with the same prefix are divided into the same partition, and each partition comprises the number of the prefixes, the number of the suffixes and suffix arrays arranged in an ascending order; a self-adaptive range filter is constructed, in the construction process, coding is carried out with partitions as units, prefixes are inserted into a cuckoo filter, suffixes are inserted into a bridge structure, suffix arrays are continuously and compactly stored according to the sparse degree, and compression processing is carried out; according to the query range, the left boundary and the right boundary are segmented into two prefix-suffix pairs, the corresponding partition is rapidly positioned according to the prefix, range query is executed in the suffix array of the partition, and the suffix meeting the condition is returned. According to the invention, the self-adaptive range query filter based on fingerprints is realized through key segmentation and a specific coding mode, the inclined workload can be adaptively coped, and the query performance is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of databases, and in particular relates to an adaptive range query filtering method, system, device and storage medium. Background Art

[0002] Key-value stores have been widely used in various applications, including time series databases, real-time analytics, big data processing, log processing, caching systems, and the Internet of Things, which require efficient handling of high write loads and scalability. Range queries are crucial for key-value stores in real-world scenarios. For example, an administrator may need to query student information with IDs ranging from 1762320 to 1762329; a user may want to retrieve a list of bills between July and August; a navigation application may need to obtain traffic conditions in a specific geographic area to plan a route. The LSM tree used in key-value stores may cause read amplification due to a large number of disk I / O operations across files. To alleviate the read amplification problem in range queries, range filters are proposed to filter out empty ranges in advance.

[0003] Existing range filters can be divided into four types: prefix tree-based, bitmap-based, hash-based, and hybrid-based. Prefix tree-based filters often have a high false positive rate due to low space efficiency; bitmap-based filters cannot handle key-query strongly correlated workloads; hash-based filters cannot guarantee the optimal false positive rate under different loads at the same time; hybrid-based methods also have performance bottlenecks.

[0004] Furthermore, existing range filters usually assume that the query distribution is uniform, which degrades the performance in the face of skewed and adversarial queries.

[0005] Therefore, the current range filter has the following problems:

[0006] (1) Existing range filters cannot adapt to adversarial queries due to the problem of bit expansion.

[0007] (2) Existing range filters lack scalability and cannot adapt to both related and unrelated workloads. Summary of the invention

[0008] Purpose of the invention: In order to optimize the false alarm rate of the range query filter under a tilted workload, the present invention implements an adaptive range query filter and proposes an adaptive range query filtering method, which can adaptively update fingerprints to avoid repeated false alarms of the same query.

[0009] Another object of the present invention is to provide an adaptive range query filtering system, device and storage medium.

[0010] In order to achieve the above invention object, the technical solution of the present invention is as follows:

[0011] An adaptive range query filtering method comprises the following steps:

[0012] Split each key in the input data set into prefix and suffix, and divide the keys with the same prefix into the same partition. Each partition includes the prefix, the number of suffixes, and the suffix array in ascending order.

[0013] An adaptive range filter is constructed, wherein the construction process is encoded in units of partitions, wherein a prefix is ​​encoded into a cuckoo filter, a fingerprint of the prefix is ​​dynamically extracted from a hash code through a sliding window and an indicator is used to represent the offset of the sliding window when the prefix is ​​inserted into a bucket of the cuckoo filter; the number of suffixes is bit-encoded and inserted into a bridge structure, wherein the bridge structure is formed by connecting the same index bits of multiple bitmaps representing a suffix tree; the suffix array is encoded into the suffix tree in a manner combining bitmap and binary representation; and the bitmap and the cuckoo filter are compressed and stored respectively;

[0014] According to the query range, the left and right boundaries are divided into two prefix-suffix pairs, and the corresponding partition is quickly located according to the prefix. A range query is performed in the suffix array of the partition to return the suffixes that meet the conditions.

[0015] Preferably, when splitting the prefix and suffix, for an L-bit key, given a maximum query range of 2l, the prefix consists of the highest Ll bits of the key, and the suffix consists of the lowest l bits of the key.

[0016] Preferably, prefixes are encoded into the cuckoo filter, including:

[0017] A hash code of the prefix is ​​obtained by using a hash function, and a fingerprint of a specified length is dynamically extracted from the hash code through a sliding window, wherein the number of bits of the specified length on the low-order side is excluded from being captured by the sliding window;

[0018] Another hash function is used to determine the insertion position of the prefix in the cuckoo filter bucket. If no conflict occurs in the target bucket, the fingerprint with the indicator is inserted; if a conflict is detected, the sliding window is moved forward by the specified length, the value of the indicator is updated plus 1, and the corresponding fingerprint is inserted.

[0019] Preferably, the number of suffixes is bit-coded and inserted into the bridge structure, including: inserting the number of suffixes bit-coded into the bitmap, if the prefix insertion position of the cuckoo filter is (i, j), i and j represent the serial number of the bucket and the serial number of the slot in the bucket respectively, and the bucket size of the cuckoo filter is b, then the insertion position of the bitmap is i*b+j.

[0020] Preferably, the insertion of the suffix array is divided into two rounds. In the first round, the partition index is inserted to represent the suffix array, where the partition index is stored using a local index table that corresponds one-to-one to the cuckoo filter; in the second round, the local index table is traversed to build a compact suffix segment, where the representation type is determined according to the corresponding number of suffixes in the bridge structure.

[0021] Preferably, the bitmap and the cuckoo filter are compressed respectively, including: using Elias-Fano coding to compress the bitmap, and using a semi-sorting technique to compress the cuckoo filter.

[0022] Preferably, the corresponding partition is quickly located according to the prefix, and a range query is performed in the suffix array of the partition to return the suffixes that meet the conditions, including:

[0023] If the left prefix is ​​equal to the right prefix, query the cuckoo filter for the left prefix. If the prefix exists, query the bridge to get its offset into the suffix tree. Find the target range in the suffix tree using binary search or bit counting, depending on its representation type. If the suffix also exists, access the cuckoo hash table to check for adaptivity before returning true.

[0024] If the left prefix is ​​different from the right prefix, then probe the respective left and right segments independently. For the left segment, locate the suffix tree as above, but only need to compare the left suffix with the largest leaf in the tree. If the left suffix is ​​less than or equal to the largest leaf, check adaptivity and return true. For the right segment, compare the right suffix with the smallest leaf in the suffix tree. If the right suffix is ​​greater than or equal to the smallest leaf, check adaptivity and return true.

[0025] An adaptive range query filtering system, comprising:

[0026] The data set preprocessing module is used to split each key in the input data set into prefix and suffix, and divide the keys with the same prefix into the same partition. Each partition includes the prefix, the number of suffixes and the suffix array in ascending order.

[0027] A range filter construction module is used to construct an adaptive range filter. The construction process is encoded in units of partitions, wherein a prefix is ​​encoded into a cuckoo filter, a fingerprint of the prefix is ​​dynamically extracted from a hash code through a sliding window and an indicator is used to represent the offset of the sliding window when inserted into the cuckoo filter bucket; the number of suffixes is bit-encoded and inserted into a bridge structure, wherein the bridge structure is formed by connecting the same index bits of multiple bitmaps representing the suffix tree; the suffix array is encoded into the suffix tree in a manner combining bitmap and binary representation; and the bitmap and the cuckoo filter are compressed and stored respectively;

[0028] The range filter query module is used to divide the left and right boundaries into two prefix-suffix pairs according to the query range, quickly locate the corresponding partition according to the prefix, perform range query in the suffix array of the partition, and return the suffixes that meet the conditions.

[0029] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the adaptive range query filtering method as described above are implemented.

[0030] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the adaptive range query filtering method described above are implemented.

[0031] Beneficial effects:

[0032] (1) The present invention implements a fingerprint-based range query filter by encoding prefixes with fingerprints, compactly storing suffixes according to sparsity, and mapping the prefixes and suffixes one by one through a bridge structure.

[0033] (2) The present invention updates the fingerprint version after a false alarm occurs to avoid repeated false alarms. On the semi-sorted version of the cuckoo filter, the present invention proposes changing the high-order fingerprint to achieve self-adaptation.

[0034] (3) The present invention samples queries and predicts query load relevance, thereby dynamically adjusting prefix and suffix memory allocation to achieve an optimal false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flow chart of an adaptive range query filtering method;

[0036] Figure 2 It is a schematic diagram of the adaptive range filter;

[0037] Figure 3 An example of inserting an adaptive range filter;

[0038] Figure 4 An example query for an adaptive range filter;

[0039] Figure 5 Schematic diagram of sliding window adaptive strategy;

[0040] Figure 6 Schematic diagram of the sliding window advanced adaptive strategy;

[0041] Figure 7Comparison results of various range filters on different query ranges and query load types. DETAILED DESCRIPTION

[0042] Reference Figure 1 The present invention proposes an adaptive range query filtering method, which comprises the following steps:

[0043] Step 1: Dataset preprocessing.

[0044] Consider the input data set as a set of keys stored in the database. For each key in the set, split it into a prefix and a suffix. For an L-bit key, given a maximum query range of 2 l , the prefix consists of the highest L1 bits, and the suffix consists of the lowest l bits. The keys with equal prefixes are divided into a partition, which consists of three parts: the prefix, the number of suffixes, and the suffix array. The number of suffixes is the number of non-repeating suffixes, and the suffixes in the suffix array are arranged in ascending order.

[0045] Step 2, construction of adaptive range filter.

[0046] The construction of the filter includes four stages: prefix insertion, bridge insertion, suffix array insertion, and compression. The construction process is inserted in units of partitions, where the prefix is ​​inserted into the cuckoo filter, the number of suffixes is inserted into the bridge structure, and the suffix array is stored continuously and compactly according to the sparsity. In the compression stage, the bitmap is compressed by Elias-Fano coding, and the cuckoo filter is compressed using semi-sorting technology.

[0047] (a) Insertion of prefix

[0048] The core idea of ​​prefix encoding is to use a compact data structure to filter out a large number of empty prefixes. The present invention uses a cuckoo filter to store prefixes because the fingerprints stored therein are only occupied by unique elements, which provides the possibility of adaptability. In addition, the positions within the filter provide the possibility of a one-to-one mapping between prefixes and suffixes. Similar to the standard cuckoo filter, only the fingerprint of the prefix is ​​stored. In order to support adaptability, a hash selector and a cuckoo hash table are used. The units in these two structures correspond one-to-one to the units in the cuckoo filter.

[0049] First, the hash code is obtained and the bucket for inserting the prefix is ​​determined using a hash function. If no collision occurs in the target bucket, the fingerprint with indicator 0 is inserted. However, if a collision is detected, the indicator value is changed in an adaptive manner and the corresponding fingerprint is inserted.

[0050] (b) Bridge insertion

[0051] In order to enable the suffix tree to be accessed in a compact data structure of variable length, multiple bitmaps are used to jointly encode the address and type information of the suffix tree, thereby ensuring efficient storage and retrieval. Leaf counts are used as bridge information to locate the suffix tree. Each bit in the bitmap corresponds to a cell in the cuckoo filter bucket. The same index bits of all bitmaps are bit-connected to form bridge information.

[0052] Insert the suffix count of the second element in the partition into the bitmap. The suffix count is inserted into different bitmaps bit by bit. Assume that the prefix insertion position in the cuckoo filter is (i, j), i is the serial number of the bucket, j is the serial number of the slot in the bucket, and the bucket size of the cuckoo filter is b, then the insertion position of the bitmap is i*b+j.

[0053] (c) Insertion of suffix

[0054] In order to achieve the best space efficiency, two different representations are selected to encode the suffix tree according to its sparsity. The space utilization is optimized by combining bitmap and binary representation.

[0055] In the bitmap representation, the length of the bitmap is aligned with the number of leaves, and each existing leaf is set to 1 to indicate its existence. SIMD instruction sets can efficiently obtain the 1-bit count in the bitmap in constant time. However, when the keys in the suffix tree are sparsely distributed, it is proven to be more space-efficient to directly store the binary representation of the key. Therefore, sparse suffixes should be encoded in binary representation. Otherwise, it is preferred to use a bitmap representation. In the binary representation, a binary search algorithm is used to find the suffix.

[0056] Insert the suffix array in two rounds. Although tuples are inserted in order, the position in the cuckoo filter is random due to the hash function. Therefore, the insertion position of the suffix array is uncertain until all prefixes and bridges are inserted. In the first round, simply insert the partition index to represent the suffix array. Use a local index table that corresponds to the cuckoo filter one-to-one to store these indexes. In the second round, traverse the local index table to build a compact suffix segment. The representation type is determined based on the corresponding suffix count in the bridge segment.

[0057] Figure 2 is an example of an adaptive range filter according to the present invention. Figure 3 The specific implementation process of inserting {12,13} is shown.

[0058] like Figure 2 As shown, in this example, we assume that L = 4, l = 2; the maximum query range is 2 l =4. 2 bits can represent a range of 4, for example, the binary representation of the range [0,1,2,3] is [00,01,10,11].

[0059] Reference Figure 3 , we also assume that the maximum query range is 4, so the length of the prefix and suffix is ​​2. Then 12 (1100) can be expressed as prefix 11 and suffix 00; 13 (1101) can be expressed as prefix 11 and suffix 01. Obviously, they have the same prefix and belong to the same partition, so the partition can be expressed as (prefix, number of suffixes, suffix array), such as Figure 3 As shown. Assume that the hash value of prefix 11 is 10100101, and then first compare it with the existing elements in the bucket to see if there is a collision. The selector of the first element is 1, so the upper four bits of the hash value 1010 are not equal to the fingerprint 0010; the selector of the second element is 0, and the lower four bits of the hash value 0101 are not equal to the fingerprint 0011; the selector of the third element is 1, and the upper four bits of the hash value 1010 are not equal to the fingerprint 1000; therefore, there is no collision, and the maximum selector used in the insertion comparison is 1, and the complete hash value is inserted into the cuckoo hash table.

[0060] The bridge module only needs to insert the suffix number 2 (010) bit by bit, 0 is inserted into the 0th bitmap, 1 is inserted into the 1st bitmap, and 0 is inserted into the 2nd bitmap.

[0061] Finally, for the insertion of the suffix array, the bitmap form of [0,1] is 1100 and the binary form is 0001, so the two forms occupy the same amount of space. Since the search speed of the bitmap is a constant complexity, the bitmap form is chosen for insertion.

[0062] (d) Compression

[0063] To improve space efficiency, Elias-Fano coding is used to efficiently compress bitmaps, and semi-sorting is applied to compress cuckoo filters. Elias-Fano coding is an algorithm for compressing sparse bitmaps that compresses data at a near-optimal rate while maintaining efficient decoding. This encoding is particularly well suited for representing sparse binary data such as 1s and 0s in a bitmap. Semi-sorting is a data compression technique that reduces storage space requirements by rearranging data elements while maintaining the relative order of the data. Applying semi-sorting in a cuckoo filter reduces storage requirements because the elements in the filter do not need to be completely sorted, but only need to remain in sufficient order to support fast lookup and insertion operations. The application of these two techniques helps optimize the use of storage space while maintaining the performance of the data structure.

[0064] Cuckoo filter compression: For a cuckoo filter with bucket size = 4 and fingerprint size = 4, one bucket occupies 16 bits. However, if the four slots in the bucket are arranged in ascending order, there are only 3876 arrangements. In fact, only 12 bits can be used to represent 4096 arrangements. Therefore, the present invention only needs 12 bits to represent one bucket.

[0065] Bitmap Compression: Elias-Fano is a form of entropy coding used to represent a monotonically non-decreasing sequence of N integers. The positions of 1s in the bitmap are treated as an array of integers. In Elias-Fano encoding, the integers are first encoded in binary using log2(M) bits if [0,M) is the full set. This representation is split into two parts: the upper log2(N) bits and the remaining log2(M / N) bits. The lower part is stored directly concatenated, using Nlog2(M / N) bits. The upper part is represented by a 2N-bit vector as described below. First, a count of the number of occurrences of all values ​​between [0,N-1] is created. This count vector is then converted to a unary representation; that is, each count is represented in a unary way (a series of 1s) and the values ​​are separated by a stop bit of 0, which results in a total of 2N bits.

[0066] Step 3: Query the adaptive range filter.

[0067] For a range query R[a,b], the adaptive range filter needs to return true or false, indicating whether the key in the query range exists in the database. The left and right boundaries a and b are divided into two prefix-suffix pairs according to the segmentation method of the data set preprocessing. First query the prefix fingerprint. If all prefix fingerprints do not match, return false directly; if there is a matching prefix fingerprint, query the corresponding suffix tree. If all suffix trees do not intersect with the query range, return false; otherwise, return true. Specifically, first compare the two prefixes. If the left prefix is ​​equal to the right prefix, only the left prefix needs to be detected in the cuckoo filter. If the prefix exists, it queries the bridge to obtain the offset of its suffix tree. Then, according to its representation type, it uses binary search or bit counting to find the target range in the suffix tree. If the suffix also exists, it will access the cuckoo hash table to check the adaptation before returning true. Checking the adaptation means checking whether the complete hash value of the prefix of the range query is consistent with the hash value corresponding to the fingerprint found. If not, it means it is a false positive, and the fingerprint needs to be updated adaptively to avoid false positives in subsequent identical queries.

[0068] If the left prefix is ​​different from the right prefix, then the respective left and right segments are probed independently. For the left segment, it locates the suffix tree as above, but only needs to compare the left suffix with the largest leaf in the tree. If the left suffix is ​​less than or equal to the largest leaf, then it checks for adaptivity and returns true. Similarly, for the right segment, the right suffix is ​​compared with the smallest leaf in the suffix tree to determine the result.

[0069] Figure 4 A query example is shown. For the range query [8,10], first get two prefix-suffix pairs: (102,0) and (102,2). Since the prefixes are the same, continue to look for 102 in the cuckoo table. Using the corresponding indicator in the hash selector, compare its hash code 10000001 with the 4 cells in the bucket. The third cell matches the fingerprint 1000 and indicator 1. Secondly, get the suffix count = 1×2 0 = 1 and offset = 2×(1×2 0 +1×2 1 )=6, so the encoded suffix tree is the binary representation 01. Third, use binary search and find that leaf 01 is in the target range [0,2]. Fourth, access the third cell in the cuckoo hash table and confirm that it is not a false positive. Therefore, no adaptation is required and true is returned.

[0070] Combine the following Figure 5 and Figure 6 The adaptive process of the range filter is described. The core idea of ​​adaptation is to dynamically update the fingerprint to defend against recurring false positives. Since the fingerprint is usually shorter than its corresponding hash code, the remaining bits of the hash code are usually redundant and unutilized. Inspired by Mitzenmacher et al., the present invention uses a sliding window to dynamically extract the fingerprint from the hash code and uses an indicator to track the offset of the sliding window. Assuming that the fingerprint length is 4 bits and the hash code is an 8-bit sequence, initially the sliding window captures the 4 least significant bits of the hash code and the indicator is 0. If a false positive is detected, the sliding window moves forward 4 bits. This will generate a new fingerprint and update the indicator to 1. When the sliding window contains the 4 most significant bits and encounters a false positive, it loops back to the initial offset. As Figure 5 As shown, when the third element 10000001 is inserted, it collides with the first element when the fingerprint version is 0. Therefore, the fingerprint versions of both are +1, the fingerprint of the first element becomes 0010, and the inserted fingerprint is 1000.

[0071] However, the semi-sorting technique used in compressing the cuckoo filter may conflict with the adaptation. To solve this problem, the present invention introduces an advanced adaptation strategy. The lowest 4 bits are excluded as the fingerprint basis, and then the above sliding window technique is applied to the remaining bits. Figure 6 As shown, assuming the fingerprint length is 8, for the hash code 100000010001, the fingerprint is 00010001 with indicator 0 or 10000001 with indicator 1.

[0072] Because the semi-sorting technology needs to ensure that the lowest 4 bits of the fingerprints in the four buckets are arranged in ascending order, the present invention does not change the lower 4 bits when updating the fingerprint, thereby avoiding disorder conflicts caused by updating the fingerprint.

[0073] Existing solutions are unable to cope with skewed query workloads, resulting in a high false alarm rate. The present invention first designs a fingerprint-based range filter, and then adds an adaptive fingerprint update strategy based on it, so as to ensure that for skewed range queries, if a false positive query appears multiple times, true will only be returned for the first query, and false will be returned after the fingerprint is adjusted. The present invention can not only adaptively cope with skewed workloads, but also maintain a lower false alarm rate under various related query workloads. Experiments have shown that the present invention can improve performance by 9.8-35.4 times under skewed query workloads. The present invention

[0074] The experimental results are as follows Figure 7 As shown, the method of the present invention is compared with the existing SOTA range filter in different query ranges and different query load types. Point queries are point queries, small range means that the maximum query range is 32, large range means that the maximum query range is 1024, Correlated means that the boundary of the query range is adjacent to the key of the data set in the database; uncorrelated means that the boundary of the query range is randomly generated. Q / U ratio refers to the ratio of the total number of queries divided by the number of non-repeated queries. It can be seen that the false alarm rate of the method of the present invention is lower.

[0075] Based on the same technical concept as the method embodiment, the present invention also provides an adaptive range query filtering system, including:

[0076] The data set preprocessing module is used to split each key in the input data set into prefix and suffix, and divide the keys with the same prefix into the same partition. Each partition includes the prefix, the number of suffixes and the suffix array in ascending order.

[0077] A range filter construction module is used to construct an adaptive range filter. The construction process is encoded in units of partitions, wherein a prefix is ​​encoded into a cuckoo filter, a fingerprint of the prefix is ​​dynamically extracted from a hash code through a sliding window and an indicator is used to represent the offset of the sliding window when inserted into the cuckoo filter bucket; the number of suffixes is bit-encoded and inserted into a bridge structure, wherein the bridge structure is formed by connecting the same index bits of multiple bitmaps representing the suffix tree; the suffix array is encoded into the suffix tree in a manner combining bitmap and binary representation; and the bitmap and the cuckoo filter are compressed and stored respectively;

[0078] The range filter query module is used to divide the left and right boundaries into two prefix-suffix pairs according to the query range, quickly locate the corresponding partition according to the prefix, perform range query in the suffix array of the partition, and return the suffixes that meet the conditions.

[0079] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the adaptive range query filtering method as described above are implemented.

[0080] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the adaptive range query filtering method described above are implemented.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An adaptive range query filtering method, characterized in that: The following steps are involved: Split each key in the input data set into prefix and suffix, and divide the keys with the same prefix into the same partition. Each partition includes the prefix, the number of suffixes, and the suffix array in ascending order. An adaptive range filter is constructed, wherein the construction process is encoded in units of partitions, wherein a prefix is ​​encoded into a cuckoo filter, a fingerprint of the prefix is ​​dynamically extracted from a hash code through a sliding window and an indicator is used to represent the offset of the sliding window when the prefix is ​​inserted into a bucket of the cuckoo filter; the number of suffixes is bit-encoded and inserted into a bridge structure, wherein the bridge structure is formed by connecting the same index bits of multiple bitmaps representing a suffix tree; a suffix array is encoded into a suffix tree in a manner combining bitmap and binary representation; The bitmap and cuckoo filter are compressed and stored separately; According to the query range, the left and right boundaries are divided into two prefix-suffix pairs, and the corresponding partition is quickly located according to the prefix. A range query is performed in the suffix array of the partition to return the suffixes that meet the conditions.

2. The method according to claim 1, characterized in that When splitting prefixes and suffixes, for an L-bit key, a maximum query range of 2 is given l , the prefix consists of the highest L1 bits of the key, and the suffix consists of the lowest L bits of the key.

3. The method according to claim 1, characterized in that Prefix encoding to cuckoo filter, including: A hash code of the prefix is ​​obtained by using a hash function, and a fingerprint of a specified length is dynamically extracted from the hash code through a sliding window, wherein the number of bits of the specified length on the low-order side is excluded from being captured by the sliding window; Another hash function is used to determine the insertion position of the prefix in the cuckoo filter bucket. If no conflict occurs in the target bucket, the fingerprint with the indicator is inserted; if a conflict is detected, the sliding window is moved forward by the specified length, the value of the indicator is updated plus 1, and the corresponding fingerprint is inserted.

4. The method according to claim 1, characterized in that: The number of suffixes is bit-coded and inserted into the bridge structure, including: inserting the number of suffixes into the bitmap in bit-coded form, if the prefix insertion position of the cuckoo filter is (i, j), i and j represent the serial number of the bucket and the serial number of the slot in the bucket respectively, and the bucket size of the cuckoo filter is b, then the insertion position of the bitmap is i*b+j.

5. The method according to claim 1, characterized in that: The insertion of the suffix array is divided into two rounds. In the first round, the partition index is inserted to represent the suffix array, where the partition index is stored using a local index table that corresponds one-to-one to the cuckoo filter; in the second round, the local index table is traversed to build a compact suffix segment, where the representation type is determined according to the corresponding number of suffixes in the bridge structure.

6. The method according to claim 1, characterized in that The bitmap and the cuckoo filter are compressed respectively, including: using Elias-Fano coding to compress the bitmap, and using a semi-sorting technique to compress the cuckoo filter.

7. The method according to claim 1, characterized in that Quickly locate the corresponding partition based on the prefix, perform a range query in the suffix array of the partition, and return the suffixes that meet the conditions, including: If the left prefix is ​​equal to the right prefix, query the cuckoo filter for the left prefix. If the prefix exists, query the bridge to get its offset into the suffix tree. Find the target range in the suffix tree using binary search or bit counting, depending on its representation type. If the suffix also exists, access the cuckoo hash table to check for adaptivity before returning true. If the left prefix is ​​different from the right prefix, then probe the respective left and right segments independently. For the left segment, locate the suffix tree as above, but only need to compare the left suffix with the largest leaf in the tree. If the left suffix is ​​less than or equal to the largest leaf, check adaptivity and return true. For the right segment, compare the right suffix with the smallest leaf in the suffix tree. If the right suffix is ​​greater than or equal to the smallest leaf, check adaptivity and return true.

8. An adaptive range query filtering system, characterized in that: include: The data set preprocessing module is used to split each key in the input data set into prefix and suffix, and divide the keys with the same prefix into the same partition. Each partition includes the prefix, the number of suffixes and the suffix array in ascending order. A range filter construction module is used to construct an adaptive range filter. The construction process is encoded in units of partitions, wherein a prefix is ​​encoded into a cuckoo filter, a fingerprint of the prefix is ​​dynamically extracted from a hash code through a sliding window and an indicator is used to represent the offset of the sliding window when the prefix is ​​inserted into the bucket of the cuckoo filter; the number of suffixes is bit-encoded and inserted into a bridge structure, wherein the bridge structure is formed by connecting the same index bits of multiple bitmaps representing the suffix tree; the suffix array is encoded into the suffix tree in a manner combining bitmap and binary representation; The bitmap and cuckoo filter are compressed and stored separately; The range filter query module is used to divide the left and right boundaries into two prefix-suffix pairs according to the query range, quickly locate the corresponding partition according to the prefix, perform range query in the suffix array of the partition, and return the suffixes that meet the conditions.

9. A computer device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the adaptive range query filtering method as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the adaptive range query filtering method according to any one of claims 1 to 7 are implemented.

Images