Cuckoo filter, data insertion method, data query method and data deletion method

Abstract

The present invention relates to the technical field of computer information representation and retrieval. Provided are a cuckoo filter, a data insertion method, a data query method and a data deletion method. The cuckoo filter is composed of a fingerprint record table and a position flag table. The fingerprint record table is composed of m buckets and is used for storing inserted data fingerprints, each bucket having b slots for storing the data fingerprints. The position flag table is composed of m vectors, with one vector being created for each bucket, and the position flag table is used for recording insertion position information of the data fingerprints in the fingerprint record table. Each data fingerprint corresponds to two candidate buckets, one slot of one bucket is selected from the two candidate buckets to store the data fingerprint, and when and only when the data fingerprint is finally stored in a second candidate bucket, a position subscript value of the slot is recorded in a vector corresponding to the second candidate bucket. The present invention has a higher data fingerprint calculation efficiency and a lower data query false positive rate, and solves the technical problem of query false positives occurring in a cuckoo filter.

Description

A cuckoo filter and data insertion, query and deletion method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The invention claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 15, 2023, with application number 202310712462.5 and invention name “A Cuckoo Filter and Data Insertion, Query and Deletion Method”, the entire contents of which are incorporated by reference into the present invention and constitute a part of the present invention for all purposes. Technical Field

[0003] The invention belongs to the technical field of computer information representation and retrieval, and in particular relates to a cuckoo filter and data insertion, query and deletion methods. Background Art

[0004] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0005] The approximate membership query data structure (AMQ) stores a probabilistic representation of a key set S on a data domain U in a compact format and supports data insertion and query operations. Some AMQs support data deletion operations. For queries on elements already in a set, it can efficiently complete set membership queries. For queries on elements outside the set, it has a controllable false positive rate (hereinafter referred to as the false positive rate), that is, when querying an element that does not exist in the set, there is a certain probability that the element exists in the set. The most important feature of AMQ is its high space efficiency. At an acceptable false positive rate, AMQ can work on devices with limited memory resources such as network routers, switches or IoT devices.

[0006] Bloom Filter (BF) is a typical example of AMQ. It supports insert and query operations on a set S of keys. If the query is for a key that exists in set S, the Bloom filter can quickly complete the query; for keys outside set S, since it is a probabilistic structure, the probability of returning "not present" is at least 1-ε, where n is the number of added elements, k is the number of hash functions used, and m is the length of the Bloom filter. This indicates that BF offers a controllable false positive rate ε, providing a trade-off between space efficiency and query accuracy. Specifically, the length of the Bloom filter directly affects the false positive rate; longer Bloom filters have a higher false positive rate. Furthermore, the number of hash functions also requires a trade-off: a larger number of hash functions results in lower Bloom filter efficiency, but too few hash functions increase the false positive rate. In recent years, BF has been widely used in packet classification, payload inspection for deep packet inspection (DPI), reducing disk I / O, avoiding database cache penetration, and data services on mobile terminals and IoT devices, including distributed joins and semi-joins, indexing, auxiliary metadata, and query processing. Its main advantage is that, given limited space, a large number of classification rules can be stored and accessed in a very compact form in dedicated hardware such as FPGAs. Over the past decade, Bloom filters have been a common solution when hardware storage is limited or frequent external memory accesses result in high latency.

[0007] Compared with ordinary hash tables or binary trees, the main advantages of BF are fixed size and constant query and insertion efficiency independent of the number of elements in the structure; the main disadvantage of BF is that it does not support data deletion operations. Counting Bloom Filter (CBF) solves the problem of BF not supporting data deletion, but it requires three to four times more space than BF to maintain the same false positive rate as BF, and once the required storage space is larger than RAM, the filter performance will drop significantly because BF uses random read and write and cannot be effectively expanded to external memory, such as flash memory; secondly, the false positive rate of the filter has been reduced to a very low level, but there is still a certain probability of false positive queries.

[0008] Recently, some scholars proposed a BF with a false positive free zone (FPFZ). By mapping elements to positions with special properties in the filter, they achieved the complete elimination of false positive rates in a given domain when the number of elements inserted into the filter is less than a given threshold. However, in all cases, the FPFZ is very small, and the supported domains and data amounts are also limited, which greatly restricts the scope of application of this technology.

[0009] Compared to BF, Cuckoo Filter (CF) supports dynamic data deletion. In terms of space efficiency, CF uses a hash function to calculate and save the fingerprint of the original collection data instead of the original data, ensuring a low false positive rate while taking up less space. In terms of time efficiency, CF uses cuckoo hashing to calculate the element insertion position, but there is a certain probability that data relocation will be required during the element insertion process due to hash collisions. A large amount of research has further optimized the insertion and query performance of the cuckoo filter, reducing the probability of relocation during element insertion and the memory usage of the filter itself. The cuckoo filter structure determines that there is a contradiction between the false positive rate and space efficiency, and the two are mutually exclusive. Therefore, the optimization of the cuckoo filter requires a comprehensive consideration of the false positive rate of its structure and the required memory space.

[0010] At present, many experts and scholars have made structural and algorithmic improvements to CF for different application scenarios. In terms of balancing the storage space and false positive rate of the cuckoo filter, it is understood that no work has eliminated the false positive rate of the cuckoo filter, and the cuckoo filter variants proposed in most works have serious efficiency problems. Therefore, further research and optimization of the structure of CF is still needed.

[0011] Summary of the Invention

[0012] To overcome the above-mentioned deficiencies in the prior art, the present invention provides a cuckoo filter and a method for data insertion, query, and deletion. The constructed cuckoo filter consists of a fingerprint record table (FRT) and a position flag table (PFT). The FCT is used to store the fingerprint of the inserted data, and the PFT is used to record the insertion position information of the data fingerprint in the fingerprint table, thereby solving the technical problem of false positive queries in the cuckoo filter.

[0013] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0014] A first aspect of the present invention provides a cuckoo filter.

[0015] A cuckoo filter is composed of a fingerprint record table and a position mark table;

[0016] The fingerprint record table consists of m buckets for storing inserted data fingerprints, wherein each bucket has b slots for storing data fingerprints;

[0017] The position mark table consists of m vectors, one vector is created for each bucket, and is used to record the insertion position information of the data fingerprint in the fingerprint record table;

[0018] Among them, each data fingerprint corresponds to two candidate buckets, and a slot of one bucket is selected to save the data fingerprint. If and only if it is finally saved in the second candidate bucket, the position subscript value of the slot is recorded in the vector corresponding to the second candidate bucket.

[0019] Furthermore, the number m of the buckets is an exponential power of 2.

[0020] Furthermore, the data fingerprint is calculated as follows:

[0021] Get the data to be inserted;

[0022] Calculate the data summary of the data to be inserted through the hash function and modulo operation: dig x =h a (x)mod2 n

[0023] Among them, n is the length of the data summary, h a (·) represents a hash function with fixed-length output;

[0024] Based on the data to be inserted and the data summary, the low-order part of the data summary is directly used as the data fingerprint of the data to be inserted.

[0025] Furthermore, the two candidate buckets are calculated as follows:

[0026] in, Represents the first candidate bucket, Indicates the second candidate bucket, x indicates the data to be inserted, dig x Indicates data summary, f x Represents data fingerprint, l is the fingerprint length, h b (·) represents a hash function with fixed-length output, and m represents the number of buckets.

[0027] A second aspect of the present invention provides a data insertion method.

[0028] A data insertion method, based on a cuckoo filter provided in the first aspect, when inserting a data x, first calculates the data digest dig corresponding to x x , data fingerprint f x and two candidate buckets Then insert, the insertion process is:

[0029] Select an empty slot e from two candidate buckets j To save the data fingerprint, if there are no empty slots in the two candidate buckets, select a slot e j , evict slot e through the eviction processj The original data fingerprint f in x' , the data fingerprint f x Save to the empty slot e j middle;

[0030] If the data fingerprint f x Finally, it is saved in the second candidate bucket, then slot e j The position index value j is stored in the bucket The corresponding vector.

[0031] Furthermore, the process selects an empty slot e of one bucket from the two candidate buckets. j To save data fingerprints, there are two specific cases:

[0032] (1) If both candidate buckets have empty slots, randomly select a bucket and store the data fingerprint in any empty slot e j ,j∈[0,b), j is the slot e j The position subscript value of ;

[0033] (2) If only one of the two candidate buckets has an empty slot, the data fingerprint is stored in any empty slot e of the bucket. j ,j∈[0,b), j is the slot e j The position subscript value.

[0034] Furthermore, if there are no empty slots in the two candidate buckets, the eviction process is as follows:

[0035] Randomly select a bucket from the two candidate buckets and randomly evict any slot e in the bucket j , data fingerprint f in j∈[0,b) x' , and f x Deposit e j Update bucket Corresponding vector The position subscript value of ;

[0036] Evicted element f x' Calculate the dual position of the current insertion position. If there is an empty slot in the bucket, f x' Insert into the empty slot and update the position index value of the corresponding vector. If there is no empty slot, repeat the eviction process until all items are stored in the filter.

[0037] A third aspect of the present invention provides a data search method.

[0038] A data search method, based on a cuckoo filter provided in the first aspect, when searching for a data y, first calculates the data summary dig corresponding to y x , data fingerprint f yand two candidate buckets Then the data fingerprint f y Match all fingerprints in these two buckets:

[0039] If the first candidate bucket Middle slot e j ,j∈[0,b)The fingerprint saved is the same as f y Match, then Search for j in the query, if it does not exist, return a successful search, otherwise return a failed search;

[0040] If the second candidate bucket Middle slot e k ,k∈[0,b)The fingerprint saved and f y Match, then Search for k in the array, and if it exists, return a success result; otherwise, return a failure result.

[0041] A fourth aspect of the present invention provides a data deletion method.

[0042] A data deletion method, based on a cuckoo filter provided in the first aspect, searches for the data fingerprint of the deleted data z when deleting the data z, and performs subsequent operations in two cases:

[0043] If the search is successful, the data fingerprint is deleted from the corresponding position in the fingerprint record table, and the position mark in the corresponding vector is deleted, and a deletion success is returned;

[0044] If the search fails, it means that the element does not exist in the filter and the deletion fails.

[0045] Furthermore, the deletion operation when the search is successful is specifically as follows:

[0046] If the data fingerprint to be deleted is in the first candidate bucket, delete the data fingerprint from the first candidate bucket;

[0047] If the data fingerprint to be deleted is in the second candidate bucket, the data fingerprint is deleted from the second candidate bucket, and the position mark in the vector corresponding to the second candidate bucket is deleted.

[0048] One or more of the above technical solutions have the following beneficial effects:

[0049] The present invention has higher data fingerprint calculation efficiency: in the data insertion stage, the low-order part of the data summary is directly used when obtaining the data fingerprint, which reduces one hash operation and reduces the data insertion delay.

[0050] The present invention has a lower false positive rate for data queries: by only recording the position subscript value stored in the second candidate bucket, the array subscript is cleverly integrated into the data fingerprint, and the fingerprint comparison length is significantly increased without increasing the fingerprint length, thereby reducing the false positive rate of the cuckoo filter by m times (m is the number of buckets in the fingerprint record table), and when the fingerprint length is 8 bits, in actual operation, the average probability of the false positive rate is 0 is 93.1%; when the fingerprint length is 12 bits, compared with 0.04% of CF, the false positive rate of this filter is 0 with a probability of 99.3%.

[0051] The present invention has a smaller space cost under the same false positive rate: under the premise of 0 false positive rate, the fingerprint length required by this filter is only 8.76993 bits, while CF requires 34.3597 bits.

[0052] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0054] FIG1 is a schematic structural diagram of a cuckoo filter according to a first embodiment.

[0055] FIG2 is a schematic diagram of the structure of the fingerprint record table of the first embodiment.

[0056] FIG3 shows a first embodiment in which the number of slots b = 2, 4, 8 and the number of barrels m = 2 15 ,2 20 ,2 25 The proportion of 1 in all elements of matrix F.

[0057] FIG4 is a schematic diagram of the structure of the position mark table of the first embodiment.

[0058] FIG5 is a flow chart of a method according to the second embodiment.

[0059] FIG6 is a flow chart of a method according to the third embodiment.

[0060] FIG7 is a flow chart of a method according to the fourth embodiment. DETAILED DESCRIPTION

[0061] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0062] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0063] Example 1

[0064] In one or more embodiments, a cuckoo filter based on data fingerprint position marking is disclosed. FIG1 is a schematic structural diagram of the cuckoo filter. As shown in FIG1 , the filter is composed of a fingerprint record table (FRT) and a position flag table (PFT). The fingerprint record table FCT is used to store the fingerprint of the inserted data, and the position flag table PFT is used to record the insertion position information of the data fingerprint in the fingerprint table.

[0065] (1) Fingerprint record form

[0066] Figure 2 is a schematic diagram of the structure of the fingerprint record table. As shown in Figure 2, the fingerprint record table FCT consists of m arrays (hereinafter referred to as buckets), where m must be a power of 2. The benefit of this requirement is that when performing an XOR operation, it can be guaranteed that the calculated index must fall within the array; each bucket has b storage units e (hereinafter referred to as slots) that can store data fingerprints f.

[0067] Before inserting, searching, or deleting data in the Cuckoo filter, the data digest, data fingerprint, and two candidate buckets corresponding to the data must be calculated. Taking the data to be inserted x as an example, first, the data digest of the data to be inserted x is calculated: dig x =h a (x)mod2 n (1)

[0068] Among them, n is the length of the data summary, h a (x) is a hash function that outputs a binary string of a specific length.

[0069] Then, based on the data to be inserted and the data summary, the low-order part of the data summary is directly taken as the data fingerprint of the data to be inserted. The formula is expressed as: f x =dig x mod2 l (2)

[0070] Where l is the fingerprint length; directly using the low-order part of the data summary as the data fingerprint of the data to be inserted reduces one hash operation, reduces the data insertion delay, and has higher data fingerprint calculation efficiency.

[0071] Finally, determine the two different candidate bucket locations for inserting data x:

[0072] Among them, h b (x) is a hash function with fixed-length output.

[0073] (2) Position Mark Table PFT

[0074] The position marker table PFT in this embodiment does not use the conventional matrix F composed of 0 and 1. m×b Instead of recording the storage location information of the data fingerprint in the fingerprint recording table FCT, by only recording the location index value stored in the second candidate bucket, the fingerprint comparison length is significantly increased without increasing the fingerprint length, thereby reducing the false positive rate of data query.

[0075] In the conventional way of thinking, 1 bit is used to represent the position of the bucket where each data fingerprint is inserted, 0 and 1 represent the p1 position and p2 position respectively, and the position marking table PFT is a matrix F composed of 0 and 1 m×b Each row occupies b bits, and the memory space required to store the position marker table PFT is mb bits. Due to the collision of the hash function, the filter load cannot reach 100%, so there are many empty slots in the fingerprint record table FRT. Since F is initialized as a zero matrix, bit 0 also indicates that the corresponding position in the fingerprint record table FRT is empty, which means that F stores a lot of invalid position information.

[0076] By observing and analyzing the characteristics of data insertion in the existing cuckoo filter, it is found that most data fingerprints will be inserted into their p1 position. Figure 3 is when the number of slots b = 2, 4, 8 and the number of buckets m = 2 15 ,2 20 ,2 25 This is the ratio of 1 to all elements of matrix F, which means that the number of 0 in matrix F is much greater than 1. In extreme cases, matrix F is approximately a sparse matrix.

[0077] Based on this conclusion, only the subscript of the data fingerprint inserted into the second candidate bucket p2 in each bucket is saved. The position marking table PFT is composed of m vectors. Each vector saves the position subscript of the data fingerprint inserted into the p2 position in the corresponding bucket in the fingerprint record table FRT. The size of each subscript is log b bits. Figure 4 is a schematic diagram of the structure of the position marking table.

[0078] Using the location information in the position tag table (PFT), during the data query process, if a matching fingerprint is found in the fingerprint record table (FRT), it is necessary to confirm in the corresponding vector of the position tag table (PFT) that the two fingerprints are from the same bucket position. In other words, whether the indexes of the two fingerprints within the bucket can be found in the corresponding vector, as shown in Figure 4. The fingerprint of the query data y is hashed to bucket[i+1], and bucket[i+1] is the p2 position of y. Assuming that the fingerprint of x in the position tag table (FRT) matches y, the index of the position within the bucket of x is found in vector[i+1], indicating that the current bucket is also the p2 position of x, which means that the entire data digest of x and y is the same, so the search is returned successful.

[0079] Example 2

[0080] In one or more embodiments, a data insertion method based on a cuckoo filter is disclosed. The cuckoo filter based on data fingerprint position mark provided in the first embodiment is used. When inserting a data x, the data digest dig corresponding to x is first calculated. x , then calculate the fingerprint f of x x and two candidate buckets The data insertion process is shown in Figure 5, and the insertion algorithm is shown in Table 1. The insertion process is divided into three cases:

[0081] (1) If two candidate buckets and There are empty slots, then a bucket is randomly selected The fingerprint f x Deposit into any empty slot e j ,j∈[0,b), if i=2, that is, data fingerprint f x Finally saved in the second candidate bucket, then store j in

[0082] (2) If there is only one bucket among the two candidate buckets If there are empty slots left in the x Store in any empty slot e of the bucket j ,j∈[0,b). If i=2, that is, data fingerprint f x Finally saved in the second candidate bucket, then store j in

[0083] (3) If there are no empty slots in both candidate buckets, and Randomly select a bucket Random eviction Any slot e j, data fingerprint f in j∈[0,b) x' And f x Deposit e j In, if equal and equal Then in the vector Insert j in equal and equal Then in the vector Delete j; if i = 2, that is, data fingerprint f x Finally saved in the second candidate bucket, then store j in

[0084] Evicted element f x' Calculate the dual position of the current insertion position If there is an empty slot in the bucket, f x' Insert the element into the empty slot and update the position index value of the corresponding vector in the PFT. If there is no empty slot, repeat the above eviction process until all items are stored in the filter; there is a maximum number of evictions MAXKICKNUM. If the eviction process exceeds MAXKICKNUM, the element insertion fails and the algorithm exits.

[0085] Table 1 Insertion algorithm

[0086] Example 3

[0087] In one or more embodiments, a data search method based on a cuckoo filter is disclosed. The cuckoo filter based on data fingerprint position mark provided in the first embodiment is used. The data search process is shown in FIG6 , and the search algorithm is shown in Table 2. When searching for a data y, the data summary dig of y is first calculated. y and data fingerprint f y , then calculate the two candidate bucket positions for y The data fingerprint f y Match all fingerprints in these two buckets:

[0088] If the bucket Middle slot e j ,j∈[0,b)The fingerprint saved is the same as f y Match, then If j does not exist, it returns a successful search, otherwise it returns a failed search.

[0089] If the bucket Middle slot e k,k∈[0,b)The fingerprint saved and f y Match, then Search for k in the array, and if it exists, return a success result; otherwise, return a failure result.

[0090] Table 2 Search algorithm

[0091] Example 4

[0092] In one or more embodiments, a data deletion method based on a cuckoo filter is disclosed. The cuckoo filter based on data fingerprint position marking provided in Example 1 is used. The deletion process is shown in FIG7 , and the deletion algorithm is shown in Table 3. When deleting a data element z, the filter is first searched for the element to be deleted z in the filter using the search method provided in Example 3, and the corresponding operation is performed based on the search result:

[0093] (1) If the search is successful, the fingerprint is deleted from the corresponding position in the FRT, and the position mark in the corresponding vector of the PFT is deleted, and the deletion is successful. Specifically:

[0094] If the data fingerprint to be deleted is in the first candidate bucket, delete the data fingerprint from the first candidate bucket;

[0095] If the data fingerprint to be deleted is in the second candidate bucket, the data fingerprint is deleted from the second candidate bucket, and the position mark in the vector corresponding to the second candidate bucket is deleted.

[0096] (2) If the search fails, it means that the element does not exist in the filter and the deletion fails.

[0097] Table 3 Deletion algorithm

[0098] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. Cuckoo filter based on data fingerprint position marking, characterized in that, It is composed of a fingerprint record table and a location mark table; The fingerprint record table is composed of m buckets for storing inserted data fingerprints, wherein each bucket has b slots for storing data fingerprints; The position mark table is composed of m vectors, and a vector is created for each bucket to record the insertion position information of the data fingerprint in the fingerprint record table; Each data fingerprint corresponds to two candidate buckets, and the calculation method of the two candidate buckets is specifically as follows: in, represents the first candidate bucket, represents the second candidate bucket, x represents the data to be inserted, dig x Indicates data summary, f x represents the data fingerprint, l is the fingerprint length, h b (·) represents a hash function with fixed-length output, m represents the number of buckets; a slot of a bucket is selected to store the data fingerprint, and the position subscript value of the slot is recorded in the vector corresponding to the second candidate bucket if and only if it is finally stored in the second candidate bucket.

2. The cuckoo filter based on data fingerprint position marking according to claim 1, characterized in that: The number m of the buckets is an exponential power of 2.

3. The cuckoo filter based on data fingerprint position marking according to claim 1, characterized in that: The data fingerprint is calculated as follows: Get the data to be inserted; The data summary of the data to be inserted is calculated through the hash function and modulus operation: you x =h a (x)mod 2 n Among them, n is the length of the data summary, h a (·) represents a hash function with fixed-length output; Based on the data to be inserted and the data summary, the low-order part of the data summary is directly used as the data fingerprint of the data to be inserted.

4. A data insertion method based on cuckoo filter, characterized in that: The cuckoo filter is a cuckoo filter based on data fingerprint position marking as described in any one of claims 1 to 3. When a data x is inserted, the data digest dig corresponding to x is first calculated. x 、Data fingerprint x and two candidate buckets Then insert, the insertion process is: Select an empty slot e from two candidate buckets j To save the data fingerprint, if there are no empty slots in the two candidate buckets, select a slot e j , evict slot e through the eviction process j The original data fingerprint f in x' , the data fingerprint f x Save to the empty slot j middle; If the data fingerprint f x Finally, it is saved in the second candidate bucket, then slot e j The position index value j is stored in the bucket The corresponding vector.

5. A data insertion method based on cuckoo filter as claimed in claim 4, characterized in that: The process of selecting an empty slot e from two candidate buckets j To save data fingerprints, there are two specific cases: (1) If both candidate buckets have empty slots, a bucket is randomly selected and the data fingerprint is stored in any empty slot e j ,j∈[0,b), j is the slot e j The position subscript value of ; (2) If only one of the two candidate buckets has an empty slot, store the data fingerprint in any empty slot e of the bucket. j ,j∈[0,b), j is the slot e j The position subscript value of .

6. A method for inserting data based on a cuckoo filter as claimed in claim 4, characterized in that: If there are no empty slots in the two candidate buckets, the eviction process is as follows: Randomly select a bucket from the two candidate buckets and randomly evict any slot e in the bucket. j , data fingerprint f in j∈[0,b) x' , and f x Deposit e j In the update bucket Corresponding vector The position subscript value of ; Evicted element f x' Calculate the dual position of the current insertion position. If there is an empty slot in the bucket, f x' Insert into the empty slot and update the position index value of the corresponding vector. If there is no empty slot, repeat the eviction process until all items are stored in the filter.

7. A data search method based on a cuckoo filter, characterized in that: The cuckoo filter is a cuckoo filter based on data fingerprint position marking as described in any one of claims 1 to 3. When searching for a data y, first calculate the data digest dig corresponding to y x 、Data fingerprint y and two candidate buckets Then the data fingerprint f y Match all fingerprints in these two buckets: If the first candidate bucket Middle slot j ,j∈[0,b)The fingerprint saved and f y Match, then Search for j in the search, if it does not exist, return a successful search, otherwise return a failed search; If the second candidate bucket Middle slot k ,k∈[0,b)The fingerprint saved and f y Match, then Search for k in , if it exists, return a success, otherwise return a failure.

8. A data deletion method based on a cuckoo filter, characterized in that: The cuckoo filter is a cuckoo filter based on data fingerprint position marking as claimed in any one of claims 1 to 3. When a data z is deleted, the data fingerprint of the deleted data z is searched, and subsequent operations are performed in two cases: If the search is successful, the data fingerprint is deleted from the corresponding position in the fingerprint record table, and the position mark in the corresponding vector is deleted, and a deletion success is returned; If the search fails, it means that the deleted data z does not exist in the filter, and the deletion fails.

9. The data deletion method based on cuckoo filter as claimed in claim 8, characterized in that: The deletion operation when the search is successful is specifically as follows: If the data fingerprint to be deleted is in the first candidate bucket, delete the data fingerprint from the first candidate bucket; If the data fingerprint to be deleted is in the second candidate bucket, the data fingerprint is deleted from the second candidate bucket, and the position mark in the vector corresponding to the second candidate bucket is deleted.

Images