Log query method and device, equipment and storage medium

A query method and log technology, applied in the field of data processing, can solve the problem of time cost and high user learning cost

Inactive Publication Date: 2021-09-07
BEIJING YOUTEJIE INFORMATION TECH
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When analyzing log data, because the initial analysis goal is not clear, it takes multiple trials and errors to analyze successfully, and each trial and error requires rewriting the program. Therefore, the time cost and user learning cost of traditional analysis methods are relatively high. high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log query method and device, equipment and storage medium
  • Log query method and device, equipment and storage medium
  • Log query method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] figure 1 It is a flow chart of a log query method in Embodiment 1 of the present invention. This embodiment is applicable to the case of log query for massive log data. The method can be executed by a log query device, which can be implemented by hardware and / or or software, and can generally be integrated into electronic devices that provide log query services. Such as figure 1 As shown, the method includes:

[0048] Step 110, when it is detected that the user inputs a target query statement, decompose the target query statement into multiple sub-query statements, and the target query statement is an SPL query statement.

[0049] In this embodiment, massive log data uploaded by users may be collected and stored. When it is detected that the user has an input operation on the query interface provided by the system, the query statement in the SPL language input by the user may be obtained as the target query statement. Among them, the SPL language is a processing lan...

Embodiment 2

[0077] figure 2 It is a flow chart of a log query method in Embodiment 2 of the present invention, and this embodiment is further refined on the basis of the foregoing embodiments. Combine below figure 2 A log query method provided in this embodiment is described, including the following steps:

[0078] 1. Collect log data in real time, and perform data aggregation processing such as field extraction and indexing on the log data, and then store the log data in blocks into the log database according to the classification information such as file name and time.

[0079] 2. When it is detected that the user enters the target query statement on the query interface provided by the system, the target query statement is decomposed into multiple simple sub-query statements by identifying the pipe character in the target query statement.

[0080] 3. According to the order of positions from left to right in the target query statement, find the first target subquery statement that ca...

Embodiment 3

[0086] image 3 It is a schematic structural diagram of a log query device in Embodiment 3 of the present invention. This embodiment is applicable to the case of performing log query on massive log data, and the device can be implemented by hardware and / or software, and generally can be integrated into an electronic device that provides log query service. Such as image 3 As shown, the device includes:

[0087] Decomposition module 310, for detecting that the user inputs a target query statement, decomposing the target query statement into a plurality of sub-query statements, the target query statement being an SPL query statement;

[0088] Type determining module 320, for determining the type of each sub-query statement according to the position of each sub-query statement in the target query statement, the type of sub-query statement includes: parallel type and non-parallel type;

[0089] The parallel retrieval module 330 is used to perform parallel retrieval on the log d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a log query method and device, equipment and a storage medium. The method comprises the following steps: when it is detected that a user inputs a target query statement, decomposing the target query statement into a plurality of sub-query statements, wherein the target query statement is an SPL query statement; according to the position of each sub-query statement in the target query statement, determining the type of each sub-query statement, wherein the types of the sub-query statements include a parallel type and a non-parallel type; performing parallel retrieval on the to-be-processed log data by using the parallel type of sub-query statements through a parallel retrieval engine to obtain a preliminary retrieval result; and performing secondary processing on the preliminary retrieval result by using non-parallel type sub-query statements through a central convergence engine to obtain a log query result matched with the target query statement. According to the technical scheme provided by the embodiment of the invention, the massive logs can be quickly queried and analyzed according to the SPL query statement without writing a complex program, and the log query result is obtained.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of data processing, and in particular, to a log query method, device, equipment, and storage medium. Background technique [0002] Currently, log data can be used in troubleshooting, monitoring, security, compliance, e-discovery, etc., and has great analytical value. However, with the advent of the big data era, the number and types of logs are increasing rapidly, and it is becoming more and more difficult to analyze log content and track down potential problems. [0003] In the prior art, the common log processing system in the industry adopts the Hadoop distributed solution, which requires users to write complex MapReduce programs, or even multiple MapReduce programs to process data analysis requirements once. When analyzing log data, since the initial analysis goal is not clear, multiple trial and error are required to analyze successfully, and the program must be rewritten for ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/242G06F16/2458G06F16/248
CPCG06F16/2433G06F16/2462G06F16/2474G06F16/248
Inventor 饶琛琳梁玫娟
Owner BEIJING YOUTEJIE INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap