A Distributed Data Filtering Method Based on Hash Ring

A technology of distributed data and filtering methods, applied in database indexing, electronic digital data processing, structured data retrieval, etc., can solve the problem of not realizing distributed data filtering, and achieve the effect of avoiding data loss and reducing false positive rate.

Active Publication Date: 2020-03-13
HUNAN ANTVISION SOFTWARE
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

After receiving the data packet and determining the data receiving terminal corresponding to the data packet according to the subscription information, this invention filters the data packet according to the data concern conditions corresponding to each data receiving terminal, and sends the obtained data filtering results to the corresponding The data receiving terminal, so as to achieve the purpose of filtering the data packets without the need of the user at the data receiving terminal side according to their own needs; however, this invention does not realize distributed data filtering

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Distributed Data Filtering Method Based on Hash Ring
  • A Distributed Data Filtering Method Based on Hash Ring
  • A Distributed Data Filtering Method Based on Hash Ring

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Such as figure 1 Shown, comprise the following steps, a kind of distributed data filtering method based on hash ring of the present invention,

[0037] Step S101: The client receives the information of the distributed deduplication cluster; including the status of the node and the Token of the node, and returns the data;

[0038] Step S102: Client interface data status request, according to the consistent hash ring Range distribution, use the Murmur3hash data filter key to obtain a hash ring position value X1, calculate the Range to which X1 belongs through the distributed filter cluster range distribution, and select the corresponding Filter nodes and use RPC to send requests to remote nodes;

[0039] Step S103: the node receives the request, and locates the corresponding filter according to the filter request sent by the RPC;

[0040] Step S104: Locating the data filter block, according to the partition request sent by the RPC, after taking the remainder of the hash...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of data filter, in particular to a Hash ring-based distributed data filter method. The method comprises the following steps: S101, receiving information of a distributed de-weighted cluster by a client; S102, client interface data condition request; S103, node reception request; S104, data filter block positioning: positioning data filter blocks of data after hash complementation according to a partition request sent by an RPC; and S105, data returning: carrying out data existence judgement on corresponding data blocks according to filter keys, returning corresponding states and returning the data. After adopting the structure, the method is capable of realizing a multi-tenant function so that the clients can add filters of established types according to the business demands; the lasting backup restoration of the filters can be realized so that the data loss can be avoided; the whole cluster is constructed on the basis of a consistent hash ring, so that the cluster is filtered to realize linear extension; and for the same filter, a plurality of sub-filters can be constructed, so that the false judgement rate can be reduced.

Description

technical field [0001] The invention relates to the technical field of data filtering, in particular to a hash ring-based distributed data filtering method. Background technique [0002] Bloom filters are widely used, such as deduplication of web page URLs, identification of spam, identification of repeated elements in sets, and query acceleration. However, the use of Bloom filters is nested in the application program, which is vulnerable to the application itself. Changes, leading to abnormal conditions such as filtering logic errors and loss of existing judgment data. [0003] Chinese invention patent application CN 104601527 A discloses a data filtering method, including: receiving the data packet sent by the data generating terminal, and determining the corresponding data receiving terminal according to the stored subscription information; according to the data filtering corresponding to each determined data receiving terminal The conditions filter the data packets to o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/215G06F16/22G06F16/25
CPCG06F16/215G06F16/2255G06F16/252
Inventor 曹志富
Owner HUNAN ANTVISION SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap