Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A conditional query method and system for distributed logs based on cuckoo filter

A conditional query and filter technology, applied in the computer field, can solve problems affecting the overall performance of the system, long query time consumption, system performance degradation, etc., to achieve the effect of avoiding lengthy time, reducing the number of conditional queries, and reducing the time consumed

Active Publication Date: 2022-04-22
湖南匡安网络技术有限公司
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the above defects or improvement needs of the prior art, the present invention provides a distributed log condition query method and system based on a cuckoo filter. The technical problem of query time consumption, and the existing method of pre-segmenting tables for the database needs to face the technical problem of system performance degradation caused by too many sub-tables, and the existing method of adding data cache in the system cannot effectively solve the condition When querying, the query requirements of multi-field condition matching, spend a lot of time, and greatly affect the technical problems of the overall performance of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A conditional query method and system for distributed logs based on cuckoo filter

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] In order to make the object of the present invention, the technical solution and advantages more clearly understood, the following in conjunction with the accompanying drawings and embodiments, the present invention will be further elaborated in detail. It should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to qualify the present invention. Further, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not constitute a conflict with each other.

[0049] The present invention provides a distributed conditional query method based on a cuckoo filter, which comprises a cold gate condition cache layer, a hot data layer, a cold data layer, and a cuckoo filter layer in a manner. The main idea is that when the system executes the conditional query, the system preferentially queries the hot data layer w...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed log condition query method based on a cuckoo filter, comprising: obtaining a condition query request sent by a client, performing data query in a pre-built hot database according to the condition query request, and judging the query Whether the total amount of data received is lower than the amount of data corresponding to the conditional query request, if so, process the conditional query request to obtain the identification string, use the identification string as the key (Key), and store data in the unpopular condition cache layer Query to determine whether there is a value (Value) corresponding to the key in the unpopular condition cache layer. If not, execute the query operation in the pre-built cold database data sub-table according to the condition query request to obtain the condition query result, and set the condition The query data result processing generates a JSON string, uses the identification string as a key, and uses the JSON string as a value to form a key-value pair, and stores the key-value pair in the unpopular condition cache layer.

Description

Technical field [0001] The present invention belongs to the field of computer technology, more particularly, relates to a distributed log condition query method and system based on a cuckoo filter. Background [0002] With the rapid popularization of Internet of Things technology, terminal devices in the field of industrial control will generate a huge amount of log data every day for security analysis and evaluation of terminal devices and statistical management. How to conveniently and quickly query the targeted conditions in the huge industrial control log data set stored for months or even years has become a common and severe problem that the industrial control management system needs to face. [0003] The existing log condition query methods include distributed database sub-database or table-splitting and then querying operations and caching data. Database sharding refers to dividing a single database into multiple databases according to the system business or the possibili...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/13G06F16/14G06F16/172G06F16/18
CPCG06F16/13G06F16/144G06F16/172G06F16/1815
Inventor 李肯立夏禹余思洋周旭刘楚波肖国庆段明星张家豪巢婉琼
Owner 湖南匡安网络技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products