Distributed log condition query method and system based on cuckoo filter

A conditional query and filter technology, applied in the computer field, can solve the problems of long query time consumption, affecting the overall performance of the system, system performance degradation, etc., achieve the effect of reducing the number of conditional queries, avoiding lengthy time, and reducing the time consumed

Active Publication Date: 2021-06-18
湖南匡安网络技术有限公司
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the above defects or improvement needs of the prior art, the present invention provides a distributed log condition query method and system based on a cuckoo filter. The technical problem of query time consumption, and the existing method of pre-segmenting tables for the database needs to face the technical problem of

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed log condition query method and system based on cuckoo filter
  • Distributed log condition query method and system based on cuckoo filter

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

[0049] The invention provides a distributed condition query method based on a cuckoo filter, which includes an unpopular condition cache layer, a hot data layer, a cold data layer and a cuckoo filter layer in an implementation manner. The main idea is that when the system executes conditional query, the system first queries the hot data layer with a small amount of data, and only when the query ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed log condition query method and system based on a cuckoo filter, which comprises the following steps of: acquiring a condition query request sent by a client, performing data query in a pre-constructed hot database according to the condition query request, and judging whether the total amount of queried data is lower than the data amount corresponding to the condition query request or not; if yes, processing the condition query request to obtain an identification character string, taking the identification character string as a key (Key), and carrying out data query in the unpopular condition cache layer to judge whether a value (Value) corresponding to the key exists in the unpopular condition cache layer; if not, executing query operation in a pre-constructed cold database data sub-table according to the condition query request to obtain a condition query result, processing the condition query data result to generate a JSON character string, taking the identification character string as a key, taking the JSON character string as a value to form a key-value pair, and storing the key-value pair in the unpopular condition cache layer.

Description

technical field [0001] The invention belongs to the field of computer technology, and more specifically relates to a distributed log condition query method and system based on a cuckoo filter. Background technique [0002] With the rapid popularization of Internet of Things technology, terminal equipment in the field of industrial control will generate a huge amount of log data every day for security analysis, evaluation and statistical management of terminal equipment. How to conveniently and quickly perform targeted conditional query in the huge industrial control log data set that exists for several months or even years has become a common and severe problem that the industrial control management system needs to face. [0003] The existing log condition query methods include pre-distributed sub-databases or sub-tables of the database and then query operations, caching of data, and the like. Database sharding refers to dividing a single database into multiple databases ac...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/13G06F16/14G06F16/172G06F16/18
CPCG06F16/13G06F16/144G06F16/172G06F16/1815
Inventor 李肯立夏禹余思洋周旭刘楚波肖国庆段明星张家豪巢婉琼
Owner 湖南匡安网络技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap