Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and system for query sharing of log data stream based on hierarchical multiplexing

A technology for sharing systems and data streams, applied in the field of query sharing of log data streams, can solve problems such as inappropriate data stream query processing, and achieve the effect of effective storage and computing resource sharing

Active Publication Date: 2019-12-03
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Obviously, blocking operations are not suitable for query processing of data streams

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for query sharing of log data stream based on hierarchical multiplexing
  • Method and system for query sharing of log data stream based on hierarchical multiplexing
  • Method and system for query sharing of log data stream based on hierarchical multiplexing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] In order to make the above-mentioned features and effects of the present invention more clear and understandable, the implementation examples are listed below and described in detail in conjunction with the accompanying drawings.

[0037] The log stream query preprocessing object is based on query operators (predicates). Each tuple in the log stream entering the system must pass through all query operators before outputting query results. Thus, each tuple is stored in the corresponding query operator queue while it is waiting to be processed by the corresponding query operator. In some cases, the number of tuples in these queues is approximately the same. For example, the projection operator only forms a new tuple with the corresponding attributes of the tuple and outputs it to the waiting queue of the next query operator, and the number of tuples does not change. In the usual data flow query processing, these intermediate processing results with the same composition ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a classification multiplexing-based log data flow query sharing method and system. The method comprises the steps of registering query operations containing a plurality of query statements to a system, and intercepting tuples in log data flow by setting a sliding window; preprocessing the query statements, extracting the query statements containing the same sub-query, and storing a query result as an intermediate result, wherein the intermediate result is shared among the query operations; and extracting the tuples with the same intermediate storage structure from the intercepted tuples, and storing the tuples by adopting a shared queue to be shared among the query operations. For log flow query storage sharing, the flexibility is provided for migration under a data sharing condition on the premise of performing effective storage and quick indexing on data of the tuples of the log flow; and for sub-query sharing, the purpose of computational resource sharing is achieved by extracting and sharing the same predicate query.

Description

technical field [0001] The invention relates to the field of query and storage of log data streams, in particular to a query sharing method and system based on hierarchically multiplexed log data streams. Background technique [0002] With the continuous development and wide application of cloud computing and big data technologies, more and more attention is paid to the log data in the cluster. On the one hand, this simple log data in the cluster is the same as the traditional log data, and it needs to be stored accordingly during the operation of the system, so as to facilitate the abnormal operation and maintenance and recovery of the system. On the other hand, the log data in the cluster The data generation method is different from the traditional log form, and is generated in the system in the form of a data flow. [0003] The processing of log data streams (hereinafter referred to as log streams) is different from traditional data streams. When processing traditional d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/18G06F16/14
CPCG06F16/148G06F16/1815
Inventor 张进东孙毓忠
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com