Distributed Bloom filter system and application method thereof

Abstract

The invention provides a distributed Bloom filter system, which comprises a main control module of the distributed Bloom filter system and a plurality of child nodes. The child nodes are Bloom filters. The child nodes form a closed loop and the main control module of the distributed Bloom filter system is disposed outside the closed loop independently. The main control module of the distributed Bloom filter system is used for finishing first hash calculation of current input data and mapping the current input data to the child nodes. The child nodes are used for finishing second hash calculation of the current input data and implementing data query of the current input data according to results of the second hash calculating. The distributed Bloom filter system and an application method thereof are fast in searching speed, high in precision, fine in augmentability, low in characteristic value storage space requirement, and particularly suitable for resource and information searching in an era of cloud computing.

Description

technical field [0001] The invention relates to computer technology fields such as information retrieval and filtering, cloud computing, and distributed computing, and in particular to a distributed Bloom filtering system and a method for using the same. Background technique [0002] With the development of Internet technology, resource sharing and data interaction need to be realized all the time. Distributed technology and cloud computing technology are the development trend and research hotspot of Internet technology today. When sharing resources, we often need to judge whether a certain table item is included in a data set, for example: in a web crawler, whether a website has been visited; for the visit statistics of a website, whether the current connection IP has come to visit And so on, which requires a resource query or information retrieval. [0003] In the information processing process of resource query, which search method to use depends on the information repre...

AI Technical Summary

Problems solved by technology

[0004] In view of this, the present invention provides a distributed Bloom filtering system and its usage method to solve the shortcoming that the single-subnode Bloom filter in the prior art cannot realize fast retrieval and query of massive data, and the solution provided by the present invention can realize Quick data query and fault handling consume less storage space, and at the same time have the advantages of efficient retrieval and easy expansion. In addition, it can also realize random plugging and unplugging of sub-nodes in the distributed Bloom filtering system, which is highly practical

Images