Hash ring-based distributed data filter method

A technology of distributed data and filtering methods, which is applied in the fields of electrical digital data processing, special data processing applications, instruments, etc., can solve the problem of not realizing distributed data filtering, and achieve the effect of avoiding data loss and reducing the rate of misjudgment.

Active Publication Date: 2016-06-08
HUNAN ANTVISION SOFTWARE
View PDF4 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

After receiving the data packet and determining the data receiving terminal corresponding to the data packet according to the subscription information, this invention filters the data packet according to the data concern conditions corresponding to each data receiving terminal, and sends the ob...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hash ring-based distributed data filter method
  • Hash ring-based distributed data filter method
  • Hash ring-based distributed data filter method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Such as figure 1 Shown, comprise the following steps, a kind of distributed data filtering method based on hash ring of the present invention,

[0037] Step S101: The client receives the information of the distributed deduplication cluster; including the status of the node and the Token of the node, and returns the data;

[0038] Step S102: Client interface data status request, according to the consistent hash ring Range distribution, use the Murmur3hash data filter key to obtain a hash ring position value X1, calculate the Range to which X1 belongs through the distributed filter cluster range distribution, and select the corresponding Filter nodes and use RPC to send requests to remote nodes;

[0039] Step S103: the node receives the request, and locates the corresponding filter according to the filter request sent by the RPC;

[0040] Step S104: Locating the data filter block, according to the partition request sent by the RPC, after taking the remainder of the hash...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of data filter, in particular to a Hash ring-based distributed data filter method. The method comprises the following steps: S101, receiving information of a distributed de-weighted cluster by a client; S102, client interface data condition request; S103, node reception request; S104, data filter block positioning: positioning data filter blocks of data after hash complementation according to a partition request sent by an RPC; and S105, data returning: carrying out data existence judgement on corresponding data blocks according to filter keys, returning corresponding states and returning the data. After adopting the structure, the method is capable of realizing a multi-tenant function so that the clients can add filters of established types according to the business demands; the lasting backup restoration of the filters can be realized so that the data loss can be avoided; the whole cluster is constructed on the basis of a consistent hash ring, so that the cluster is filtered to realize linear extension; and for the same filter, a plurality of sub-filters can be constructed, so that the false judgement rate can be reduced.

Description

technical field [0001] The invention relates to the technical field of data filtering, in particular to a hash ring-based distributed data filtering method. Background technique [0002] Bloom filters are widely used, such as deduplication of web page URLs, identification of spam, identification of repeated elements in sets, and query acceleration. However, the use of Bloom filters is nested in the application program, which is vulnerable to the application itself. Changes, leading to abnormal conditions such as filtering logic errors and loss of existing judgment data. [0003] Chinese invention patent application CN104601527A discloses a data filtering method, including: receiving the data packet sent by the data generating terminal, and determining the corresponding data receiving terminal according to the stored subscription information; according to the determined data filtering conditions corresponding to each data receiving terminal The data packets are filtered to o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/215G06F16/2255G06F16/252
Inventor 曹志富
Owner HUNAN ANTVISION SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap