An information search system based on a variable fingerprint cuckoo filter

By introducing a variable fingerprint cuckoo filter into the information retrieval system, the fingerprint length and neighbor node cooperation are dynamically adjusted, solving the problems of slow information retrieval speed and low memory utilization in the existing technology, and realizing efficient and accurate information retrieval.

CN116126928BActive Publication Date: 2026-01-02ZHENGZHOU SEANET TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111334205.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-11
Publication Date
2026-01-02
Estimated Expiration
2041-11-11

AI Technical Summary

Technical Problem

Existing information retrieval methods are insufficient in terms of query speed and memory efficiency in massive datasets. In particular, the cuckoo filter has low memory utilization and high false alarm rate when the storage state changes dynamically.

Method used

An information retrieval system based on a variable fingerprint cuckoo filter is adopted. By deploying a variable fingerprint cuckoo filter on the service node, the fingerprint length in the storage bucket is dynamically adjusted to adapt to changes in network load. Multiple types of fingerprints are generated through multiple hash functions, and combined with neighbor node collaborative retrieval, fast and accurate information retrieval is achieved.

Benefits of technology

It improves the accuracy of information retrieval and memory utilization efficiency, reduces the false positive rate, and achieves efficient information retrieval under dynamic load conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116126928B_ABST
    Figure CN116126928B_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of information search, in particular to a variable fingerprint cuckoo filter-based information search system, which comprises a network user and a plurality of service nodes, wherein the service nodes with the same function and the network distance within one hop range are neighbor nodes of each other, and the data information search function is provided based on the variable fingerprint cuckoo filter deployed in each service node according to the requirement of the network user; the variable fingerprint cuckoo filter comprises a plurality of storage buckets occupying the same memory space, and each storage bucket stores a plurality of fingerprints of data information, wherein the length of the fingerprint is dynamically adjusted according to the load of the network service node. The service nodes in the system use the above technical solution to search information, which can improve the memory utilization, realize the rapid search of information, and improve the accuracy of search.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information search, and in particular to an information search system based on a variable fingerprint cuckoo filter. BACKGROUND

[0002] With the increasing requirements of Internet applications on the network, the traditional TCP / IP Internet architecture designed on the principle of end-to-end transmission has been difficult to adapt to the development trend of future networks, especially difficult to meet the development needs of new applications such as mobile access, Internet of Things, distributed information processing, and massive streaming media. In order to solve the contradiction between the architecture and the application requirements, researchers have proposed an information-centric network to build a network architecture centered on information / content, decoupling information names from network locations, targeting efficient distribution and acquisition of content, enhancing network scalability, mobility, security, and dynamicity, and realizing information interconnection of networks.

[0003] Since a single server is insufficient to store all data content, and the query speed from a single server is slow, a distributed multi-node architecture is used to store all data in fragments, store them on multiple servers according to the load, and query in parallel to improve efficiency. This requires maintaining the location information of the fragments and the communication between multiple nodes. Since each node stores a lot of information, the amount of information is huge, the types of information are diverse, the structure is complex, and the length is not limited. Therefore, it is crucial for the information search method to quickly and accurately search for information in a massive set with low memory usage.

[0004] There are several common information search methods in network nodes:

[0005] Hash table-based information search method. First, the information is calculated by a hash function to obtain a key value, and then all key values are stored in a hash table to obtain the requested content by searching the key value position in the table. Although the query speed is fast and the accuracy is high, it requires a large amount of memory.

[0006] Bloom filter-based information search method. Bloom filter is essentially composed of a bitmap, which uses m bits to represent n members of a set, and maps each inserted member to k bits of the bitmap using k hash functions, which are set to 1. Therefore, it is a space-efficient randomized data structure. By mapping the information to be searched to the corresponding bit using multiple hash functions, efficient query speed is guaranteed. However, different information may be mapped to the same bit, which will cause false positive errors, and it cannot count the number of different hash keys, so it does not support removing existing members.

[0007] The information search method based on the counting Bloom filter. The counting Bloom filter is similar to the Bloom filter, but uses m counters to represent n elements of a set to support deletion, so its space overhead is high, usually more than 4 times that of the Bloom filter.

[0008] The information search method based on the cuckoo filter. The information is stored in the cuckoo filter through the information fingerprint calculated by the hash function, instead of directly storing the information itself, which saves a lot of space. When querying, the possible storage bucket position of the information to be searched is calculated through the hash function, and the fingerprint is searched in the storage bucket to ensure O(1) search speed. False positives may occur when different information has the same fingerprint, and the false positive rate is related to the length f of the fingerprint. However, due to the dynamic changes in the storage state, the cuckoo filter usually needs 25% additional memory, and in the worst case, 50% of the memory may be in an idle state. SUMMARY

[0009] In view of the problems existing in the prior art information search method, the purpose of the present application is to overcome the above-mentioned defects of the prior art, and to provide an information search system based on a variable fingerprint cuckoo filter.

[0010] In order to achieve the above-mentioned purpose, the present application provides an information search system based on a variable fingerprint cuckoo filter, which comprises a network user and a plurality of service nodes, the service nodes having the same function and being adjacent to each other within one hop range,

[0011] According to the requirements of the network user, the data information search function is provided based on the variable fingerprint cuckoo filter deployed in each service node; the variable fingerprint cuckoo filter comprises a plurality of storage buckets occupying the same memory space, each storage bucket storing a plurality of fingerprints of data information, and the length of the fingerprint is dynamically adjusted according to the load of the network service node.

[0012] As an improvement of the above-mentioned system, each information is stored in a candidate storage bucket of the variable fingerprint cuckoo filter of a certain service node, the number of candidate storage buckets is not less than two, the number of the first candidate storage bucket is calculated by a hash function on the information, and the number of the other candidate storage buckets is calculated by XOR operation on the information or the fingerprint of the information and the number of the first candidate bucket.

[0013] As an improvement of the above-mentioned system, the structure of the storage bucket comprises a flag field and a fingerprint field; wherein,

[0014] The mark field includes a first mark bit and a second mark bit. The first mark bit is used to identify the number n of currently actually stored fingerprints in the bucket, and n does not exceed a preset storage upper limit N. The second mark bit is represented by binary coding and is used to identify the fingerprint types corresponding to the n fingerprints in the fingerprint field. The fingerprint types are in one-to-one correspondence with the hash functions used when the fingerprints are generated.

[0015] The fingerprint field is used to store the n fingerprints. When the number n of fingerprints dynamically changes from 1 to N, the length of the stored fingerprints changes correspondingly and always completely fills the storage space of the fingerprint field.

[0016] As an improvement of the above system, the fingerprint is an information digest obtained by calculating information by using a hash function, is in the form of a string, and the length of the fingerprint depends on the hash function used. Different hash functions are used to obtain different types of fingerprints.

[0017] As an improvement of the above system, the hash function used for generating the candidate bucket number is independent of the hash function used for generating the fingerprint.

[0018] As an improvement of the above system, the service node includes an information searching module, an information inserting module and an information deleting module. Wherein,

[0019] The information searching module is used to search for the fingerprint corresponding to the data information from the service node or other service nodes according to the search requirement of the network user.

[0020] The information inserting module is used to generate a fingerprint for the data information to be added and insert the fingerprint into the corresponding storage bucket, and modify the mark field of the storage bucket.

[0021] The information deleting module is used to find the corresponding storage bucket according to the data information to be deleted, delete the corresponding fingerprint, and modify the mark field of the storage bucket.

[0022] As an improvement of the above system, the specific processing process of the information searching module includes:

[0023] Step A1) searches for the information to be searched in the variable fingerprint cuckoo filter of the local service node. If the information is found, step A4) is performed. If the information is not found, the search number is increased by 1, and it is determined whether the search number exceeds a threshold value. If the search number does not exceed the threshold value, step A2) is performed. If the search number exceeds the threshold value, step A3) is performed.

[0024] Step A2) searches whether there is a neighbor node having the information to be searched in the information and neighbor node matching relationship pre-established in the local service node. If yes, the neighbor node is set as a new service node, and step A1) is performed. If no, step A3) is performed.

[0025] Step A3) The remote service nodes work together to search until the search result is obtained, then proceed to step A4);

[0026] Step A4) Output the search results.

[0027] As an improvement to the above system, step A1) involves searching the variable fingerprint cuckoo filter of the local service node based on the information to be searched; specifically, it includes:

[0028] Based on the information to be searched, candidate storage buckets corresponding to the information to be searched and several types of fingerprints corresponding to the information to be searched are determined based on multiple different hash functions.

[0029] Read the flag field in the candidate storage bucket to obtain the number of fingerprints, fingerprint length, and type information stored in the storage bucket;

[0030] The fingerprints of several types of information to be searched are compared with the fingerprints in the candidate storage bucket. If a matching fingerprint is found, the information to be searched is found; otherwise, the information to be searched is not found.

[0031] As an improvement to the above system, the specific processing procedure of the information insertion module includes:

[0032] Step B1) For the information to be inserted, calculate and determine the number of each candidate storage bucket, and calculate several types of fingerprints corresponding to the information to be inserted;

[0033] Step B2) Read the first flag bit n of the flag field in each candidate storage bucket in turn, and determine whether n is equal to N. If n is equal to N in each candidate storage bucket, go to step B4); otherwise, go to step B3).

[0034] Step B3) Insert the information fingerprint into the fingerprint field of any candidate storage bucket where n is less than N. Change the first flag bit of the candidate storage bucket to n+1, modify the original fingerprint type stored in the candidate storage bucket accordingly to completely fill the fingerprint field storage space, and modify the fingerprint type of the current n+1 fingerprints in the storage bucket corresponding to the second flag bit. Insertion is successful.

[0035] Step B4) Randomly select a candidate storage bucket, randomly select a fingerprint from it to be recorded as the victim, replace it with the fingerprint of the information to be inserted, and increment the execution count by 1;

[0036] Step B5) Treat the victim as the new information to be inserted, and proceed to step B1);

[0037] Step B6) Exit the information insertion module when there are no victims or the number of executions reaches the preset threshold.

[0038] As an improvement of the above system, the specific processing procedure of the information deletion module comprises:

[0039] Step C1) for the information to be deleted, determining the candidate storage buckets corresponding to the information and the type fingerprints corresponding to the information to be deleted based on a plurality of different hash functions;

[0040] Step C2) reading the flag field in the candidate storage bucket to determine the number of fingerprints stored in the storage bucket, the length of the fingerprints and the type information;

[0041] Step C3) comparing the fingerprints of the information to be deleted with the fingerprints in each candidate storage bucket respectively, if there is a consistent fingerprint, deleting it, if there are still fingerprints of other information stored in the candidate storage bucket, modifying the length and type of the remaining fingerprints in the candidate storage bucket, filling the fingerprint field, modifying the flag field, deleting successfully, and exiting the deletion module;

[0042] Step C4) if the fingerprint of the information to be deleted does not match the fingerprints in all candidate storage buckets, the information to be deleted does not exist, and the deletion module is exited.

[0043] Compared with the prior art, the advantages of the present application are that:

[0044] 1、The variable fingerprint cuckoo filter structure designed in the present application can flexibly modify the fingerprint of information according to the storage load condition during information searching;

[0045] 2、Using the technical scheme of the present application in the information query service node can efficiently utilize memory resources, realize fast searching of information, and improve the accuracy of searching;

[0046] 3、The system of the present application effectively solves the problems of high false positive rate and low memory efficiency of the information searching method. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a network processing schematic diagram of a user U searching for information x in a service node S in a distributed system in the embodiment of the present application;

[0048] Figure 2 is a structural schematic diagram of the variable fingerprint cuckoo filter in the embodiment of the present application;

[0049] Figure 3 is an information searching flowchart in a distributed system in the embodiment of the present application. DETAILED DESCRIPTION

[0050] The application discloses an information search system based on a variable fingerprint cuckoo filter. The system comprises network users and a plurality of service nodes, and the service nodes with the same function and within one hop of network distance are neighbor nodes of each other. The service node is a network device, and the specific functions include:

[0051] (1) capable of storing and maintaining data information;

[0052] (2) capable of storing and maintaining the matching relationship between information and neighbor nodes;

[0053] (3) capable of providing information adding, deleting and searching functions;

[0054] (4) capable of maintaining a specific part of information according to a pre-agreed method, wherein the pre-agreed method includes but is not limited to table lookup and function mapping.

[0055] The neighbor node is a network node with the same function as the current service node and belonging to the same network. The node within one hop of the current service node and responding to the broadcast message and establishing a connection becomes the neighbor of the current node. The remote service node is a service node outside the one hop of the current service node and can ensure mutual communication with the current service node.

[0056] According to the requirements of the network user, the variable fingerprint cuckoo filter deployed on each service node provides a data information search function. The variable fingerprint cuckoo filter comprises a plurality of storage buckets occupying the same memory space, each storage bucket stores a plurality of fingerprints of data information, and the fingerprint dynamically adjusts the length according to the load of the network service node. When the load of the variable fingerprint cuckoo filter changes, the stored information fingerprint dynamically changes the length to fill the storage bucket. The lower the load, the longer the average fingerprint length used, and the longer fingerprint carries more information digest, thereby reducing the false positive rate of information search.

[0057] The service node comprises an information search module, an information insertion module and an information deletion module.

[0058] The information search system based on the variable fingerprint cuckoo filter involves a plurality of mutually cooperative service nodes, and the search process is divided into three stages. The first stage is to directly search for information in a single service node locally. The second stage is to search for neighbors capable of successfully searching for the information in the adjacent nodes as new service nodes to re-enter the first stage for searching. The third stage is to allow remote service nodes to cooperatively search until the search result is obtained when the second stage fails to find the neighbors or the search times of the above two stages exceed a certain threshold.

[0059] In the first stage, local information lookup is performed within the service node by a variable-fingerprint cuckoo filter storing local information digest;

[0060] In the second stage, neighbor service node lookup is performed within the service node by a variable-fingerprint cuckoo filter storing information matching relationship with neighbor nodes.

[0061] The variable-fingerprint cuckoo filter is a new type of cuckoo filter with changeable fingerprints, which supports information insertion, lookup and deletion. The variable-fingerprint cuckoo filter includes a plurality of storage buckets, wherein the stored elements are fingerprints of a plurality of information. The variable-fingerprint cuckoo filter can quickly and accurately determine whether a given information exists in an information set. In the query, the candidate storage bucket corresponding to the information to be queried is obtained by a hash function, and the flag field of the storage bucket can quickly determine the number and type of stored fingerprints. The information is calculated by multiple hash functions to obtain multiple types of fingerprints. If the fingerprints match the fingerprints in the candidate storage bucket, the lookup is successful, otherwise the lookup fails.

[0062] The information lookup process is as follows:

[0063] Step A1: Given an information for query, determine the candidate storage bucket corresponding to the information and a plurality of types of fingerprints corresponding to the information based on a plurality of different hash functions;

[0064] Step A2: Read the flag field in the candidate storage bucket to determine the number, length and type of fingerprints stored in the storage bucket;

[0065] Step A3: Compare the fingerprints of the information to be queried with the fingerprints in the candidate storage bucket one by one. If there is a consistent fingerprint, the information is found;

[0066] Step A4: If the fingerprints of the information to be queried do not match all the fingerprints in the candidate storage bucket, the information cannot be found.

[0067] The hash function is a function that can compress an input message of arbitrary length into an output message digest of a certain fixed length. Different hash functions can compress the same input message into different output digests of equal or unequal length. The hash function includes but is not limited to MurmurHash, BobHash, SuperFastHash, MD5Hash and SHA1Hash, etc.

[0068] The candidate storage buckets are storage buckets storing or possibly storing the fingerprints of the given information, the number of candidate storage buckets corresponding to each information is not less than two, the number of the first candidate storage bucket is calculated by a hash function on the information, and the hash function used is independent of the hash function used for calculating the fingerprints; the calculation method of the number of other candidate storage buckets includes but is not limited to performing exclusive or calculation on the number of the first candidate bucket using the information itself or a certain type of fingerprint of the information.

[0069] The specific process of information insertion includes:

[0070] Step B1: For the information to be inserted, determine the candidate storage buckets corresponding to the information and the fingerprints of several types corresponding to the information based on multiple different hash functions;

[0071] Step B2: Read the flag field in the candidate storage bucket, check whether all the candidate storage buckets have reached the upper limit of the number of stored fingerprints, if all have reached the upper limit, execute step B4; otherwise, execute step B3;

[0072] Step B3: Insert the information fingerprint into the fingerprint field of any one of the candidate storage buckets with n less than N, change the first flag bit of the candidate storage bucket to n+1, correspondingly modify the type of the original stored fingerprint of the candidate storage bucket to completely fill the storage space of the fingerprint field, and modify the second flag bit corresponding to the fingerprint type of the current n+1 fingerprints in the storage bucket, and the insertion is successful;

[0073] Step B4: Arbitrarily select one of the candidate storage buckets, randomly select a fingerprint from it as a victim, and replace it with the same type of fingerprint corresponding to the information to be inserted;

[0074] Step B5: Take the victim as new information to be inserted, execute step B1 until there is no victim or the execution times reach a preset threshold.

[0075] The specific process of information deletion includes:

[0076] Step C1: For the information to be deleted, determine the candidate storage buckets corresponding to the information and the fingerprints of several types corresponding to the information based on multiple different hash functions;

[0077] Step C2: Read the flag field in the candidate storage bucket, determine the number of fingerprints stored in the storage bucket, the length and type of the fingerprints, and other information;

[0078] Step C3: Compare the fingerprints of the information to be deleted with the fingerprints in the candidate storage bucket one by one, if there is a consistent fingerprint, delete one of them, if the candidate storage bucket still stores fingerprints, modify the length and type of the remaining fingerprints in the candidate storage bucket to fill the fingerprint field, and correspondingly modify the flag field, and the deletion is successful;

[0079] Step C4: If the information fingerprint to be deleted does not match the fingerprints in all candidate storage buckets, the information does not exist, and the deletion operation is not needed.

[0080] After the service node completes the information insertion or deletion operation, it needs to send a notification to the neighbor nodes to complete the synchronization of information update.

[0081] The technical solutions of the present application will be described in detail below in combination with the drawings and embodiments.

[0082] Embodiment 1

[0083] Figure 1 A network processing schematic diagram of the present application is provided to describe the specific embodiments by taking a case where a network user U requests a service node S to find a name as an example. In the embodiment, the name x represents the information to be found by the user U, and includes the following steps:

[0084] A101: The user U sends a request to the service node S to find the name x;

[0085] A102: After receiving the request packet sent by the user U, the service node S starts to find the name x locally, and finds that the name x is not found through the variable fingerprint cuckoo filter of the node S;

[0086] A103: The service node S starts to query whether there is a neighbor that can find the information in the local neighbor nodes, and finds that the name x is not found through the variable fingerprint cuckoo filter of the neighbor S1, and the name x is found through the variable fingerprint cuckoo filter of the neighbor S2; the service node S learns that the neighbor S2 can process the request to find the name x;

[0087] A104: The service node S forwards the request to the found neighbor S2 at this time;

[0088] A105: The neighbor S2 starts to find the name x locally, and finds that the name x exists in the local cache through the variable fingerprint cuckoo filter;

[0089] A106: The address IP1 corresponding to the name x is obtained from the local cache of the neighbor S2, and is encapsulated into a response packet;

[0090] A107: The neighbor S2 sends the response packet containing the found result IP1 to the user U, and the query ends.

[0091] The process of using the variable fingerprint cuckoo filter to query in the embodiment will be described in detail below.

[0092] Figure 2The structure of the variable fingerprint cuckoo filter is shown, and in one implementation of the embodiment, the variable fingerprint cuckoo filter includes m storage buckets numbered from 1 to m, each of which stores up to 4 fingerprints of length f. The variable fingerprint cuckoo filter allows the same name to be calculated as 4 kinds of fingerprints of different lengths and types by using 4 hash functions. Take x as an example, after being operated by 4 hash functions h0(x), h1(x),..., h3(x), 4 independent fingerprints f0(x),..., f3(x) are obtained, and the lengths of the 4 fingerprints are f, f, 2f and 4f respectively. At the same time, the first candidate storage bucket position p1(x) is obtained by operating a hash function hp(x), and the second candidate storage bucket position is calculated by performing XOR operation between the first candidate storage bucket position and the name x, that is, p2(x) = p1(x) XOR x.

[0093] Figure 3 The information search flowchart in the distributed system is shown, and in one implementation of the embodiment, the query process of the variable fingerprint cuckoo filter when searching for a name x is as follows:

[0094] B101: The first candidate storage bucket position p1(x) = 2 is obtained by operating a hash function hp(x), and the second candidate storage bucket position p2(x) = m is obtained by performing XOR operation between the first candidate storage bucket position and the name x.

[0095] B102: Four fingerprints f0(x), f1(x), f2(x) and f3(x) corresponding to the name x are calculated.

[0096] B103: The flag field in the first candidate storage bucket is read, the flag of the number of stored fingerprints is 3, indicating that three fingerprints are stored in the bucket, and the flag of the type of stored fingerprints is 210, indicating that the types of the three stored fingerprints are f2, f1 and f0 respectively.

[0097] B104: The three fingerprints f2(k), f1(j) and f0(n) in the first storage bucket are read in sequence.

[0098] B105: The fingerprints of the corresponding type of x are compared with the read fingerprints, and the results are f2(x) ≠ f2(k), f1(x) ≠ f1(j) and f0(x) ≠ f0(n), so the search for x in the first storage bucket fails.

[0099] B106: The flag field in the second candidate storage bucket is read, the flag of the number of stored fingerprints is 4, indicating that four fingerprints are stored in the bucket, and the flag of the type of stored fingerprints is 0000, indicating that the types of the four stored fingerprints are f0.

[0100] B107: Read the four fingerprints f0(w), f0(z), f0(y), f0(x) in the second storage bucket in sequence;

[0101] B108: Compare the fingerprint of the x type with the read fingerprints, find f0(x)=f0(x), so find x successfully in the second storage bucket;

[0102] B109: Find the fingerprint matching x in the two candidate storage buckets, so the query is successful.

[0103] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the examples, those skilled in the art should understand that modifications or equivalent replacements of the technical solutions of the present application do not deviate from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A variable-fingerprint cuckoo filter based information lookup system, the system comprising a network user and a plurality of service nodes, wherein the service nodes with the same function and the network distance within one hop range are neighbor nodes of each other, characterized in that, according to the requirements of the network user, a data information lookup function is provided based on the variable-fingerprint cuckoo filter deployed in each service node; the variable-fingerprint cuckoo filter comprises a plurality of storage buckets occupying the same memory space, each storage bucket storing a plurality of fingerprints of data information, and the length of the fingerprint is dynamically adjusted according to the load of the network service node where the fingerprint is located to fill the storage bucket, the lower the load, the longer the average length of the fingerprint used, and the more information digest carried in the longer fingerprint, thereby reducing the false positive rate of information lookup; the structure of the storage bucket specifically comprises a flag field and a fingerprint field; wherein, the flag field comprises a first flag bit and a second flag bit, the first flag bit is used to identify the number n of fingerprints currently actually stored in the storage bucket, and n does not exceed a preset storage upper limit N; the second flag bit is encoded in binary and is used to identify the fingerprint types corresponding to the n fingerprints in the fingerprint field, and the fingerprint type is in one-to-one correspondence with the hash function used when the fingerprint is generated; the fingerprint field is used to store n fingerprints, and when the number n of fingerprints dynamically changes from 1 to N, the length of the stored fingerprint changes accordingly and always completely fills the storage space of the fingerprint field. Each of the information is stored in a candidate storage bucket of the variable-fingerprint cuckoo filter of a certain service node, the number of the candidate storage buckets is not less than two, the number of the first candidate storage bucket is calculated by a hash function on the information, and the number of the other candidate storage buckets is calculated by performing XOR operation on the information or the fingerprint of the information and the number of the first candidate bucket.

2. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 1, wherein, The fingerprint is an information digest obtained by calculating the information through a hash function, is in the form of a string, and the length of the fingerprint depends on the used hash function, different hash functions are used to obtain different types of fingerprints.

3. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 1, wherein, The hash function used for generating the number of the candidate storage bucket is independent of the hash function used for generating the fingerprint.

4. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 3, wherein, The service node comprises an information lookup module, an information insertion module and an information deletion module; wherein, 5. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 4, wherein, the information lookup module is used to find the fingerprint corresponding to the data information from the service node or other service nodes according to the lookup requirements of the network user; the information insertion module is used to generate the fingerprint of the data information to be added and insert it into the corresponding storage bucket, and modify the flag field of the storage bucket; the information deletion module is used to find the corresponding storage bucket according to the data information to be deleted, delete the corresponding fingerprint, and modify the flag field of the storage bucket. The specific processing process of the information lookup module comprises:

6. The variable-fingerprint-based cuckoo filter information lookup system according to claim 5, wherein, Step A1) searching the variable-fingerprint cuckoo filter in the local service node according to the information to be searched, if found, turning to step A4), if not found, increasing the search times by 1, and judging whether the search times exceed a threshold, if not, turning to step A2); if not found and the search times exceed the threshold, turning to step A3); ​ Step A2) whether there is a neighbor node with the information to be found in the information pre-established by the local service node and matched with the neighbor node, and if yes, the neighbor node is set as a new service node, and the process goes to step A1); otherwise, the process goes to step A3); Step A3) cooperative search by the remote service node until the search result is obtained, and the process goes to step A4); Step A4) output the search result.

7. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 6, wherein, The step A1) includes: According to the information to be found, a plurality of different hash functions are used to determine the candidate storage buckets corresponding to the information to be found and a plurality of type fingerprints corresponding to the information to be found; The flag field in the candidate storage bucket is read to obtain the number, length and type information of the fingerprints stored in the storage bucket; The plurality of type fingerprints of the information to be found are compared with the fingerprints in the candidate storage bucket respectively, and if there is a consistent fingerprint, the information to be found is found; otherwise, the information to be found is not found.

8. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 5, wherein, The specific processing process of the information insertion module includes: Step B1) for the information to be inserted, the number of each candidate storage bucket is determined, and a plurality of type fingerprints corresponding to the information to be inserted are calculated; Step B2) the first flag bit n of the flag field in each candidate storage bucket is read in sequence, and it is judged whether n is equal to N, if n of each candidate storage is equal to N, the process goes to step B4); otherwise, the process goes to step B3); Step B3) the information fingerprint is inserted into the fingerprint field of any candidate storage bucket with n less than N, the first flag bit of the candidate storage bucket is changed to n+1, the type of the original stored fingerprint of the candidate storage bucket is modified to completely occupy the storage space of the fingerprint field, and the second flag bit corresponding to the current n+1 fingerprints of the storage bucket is modified, and the insertion is successful; Step B4) randomly select a candidate storage bucket, and randomly select a fingerprint from the candidate storage bucket as a victim, replace the fingerprint with the fingerprint of the information to be inserted, and execute the number plus 1; Step B5) the victim is taken as new information to be inserted, and the process goes to step B1); Step B6) when there is no victim or the execution number reaches a preset threshold, the information insertion module is exited.

9. The variable-fingerprint-based cuckoo filter system for information lookup according to claim 5, wherein, The specific processing process of the information deletion module includes: Step C1) for the information to be deleted, a plurality of different hash functions are used to determine the candidate storage buckets corresponding to the information and a plurality of type fingerprints corresponding to the information to be deleted; Step C2) the flag field in the candidate storage bucket is read to determine the number, length and type information of the fingerprints stored in the storage bucket; Step C3) the fingerprints of the information to be deleted are compared with the fingerprints in each candidate storage bucket respectively, if there is a consistent fingerprint, the information is deleted, if the candidate storage bucket still stores the fingerprints of other information, the length and type of the remaining fingerprints in the candidate storage bucket are modified, the fingerprint field is filled, the flag field is modified, the deletion is successful, and the deletion module is exited; Step C4) if the information fingerprint to be deleted does not match the fingerprints in all candidate storage buckets, the information to be deleted does not exist, and the deletion module is exited.

Citation Information

Patent Citations

  • Workload adaptation method for measuring flow of network data stream

    CN102833134A

  • Label cuckoo filter

    CN111552693A