Calculation method and device for platform-based log analysis

A computing method and computing device technology, applied in the direction of hardware monitoring, etc., can solve problems such as waste of computing and storage resources, ignorance of each other, waste of computing and storage resources, etc., and achieve the goal of avoiding redundant computing and redundant storage, and optimizing computing methods Effect

Active Publication Date: 2018-06-29
ADVANCED NEW TECH CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, a defect brought about by platformization is the waste of computing and storage resources. Two users may have very similar requirements but do not know the existence of the other party. Most of the calculation content is repeated; However, the entire database is used as the input object for traversal during calculation. These phenomena lead to serious waste of calculation and storage resources.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Calculation method and device for platform-based log analysis
  • Calculation method and device for platform-based log analysis
  • Calculation method and device for platform-based log analysis

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056]The present invention will be described in detail below in conjunction with various embodiments shown in the drawings. However, these embodiments do not limit the present invention, and any structural, method, or functional changes easily made by those skilled in the art based on these embodiments are included in the protection scope of the present invention.

[0057] If each high-speed toll station records passing vehicle information into a log, we can analyze the log to obtain, for example, the traffic flow of each toll station, whether the G25 high-speed traffic is too high, the proportion of each source, the total number of vehicles entering Shanghai during the National Day, Various valuable information such as the escape route of the suspect's vehicle in Zhejiang A123XX. In the process of log analysis, the original log is first parsed to obtain structured data:

[0058] time

license plate

toll

high speed

city

TOLL

201211‐1111:11 ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention provides a computing method and device for platform-based log analysis, wherein the method includes: receiving a first condition group to form a first calculation rule; receiving a second condition group to form a second calculation rule; if the second calculation rule The corresponding expected result set is a subset of the result set corresponding to the first calculation rule, and the result set of the first calculation rule is used as the input data matched by the second calculation rule. Compared with the prior art, the beneficial effect of the present invention is that different calculation rules can be formed according to different condition groups, and the calculation method can be optimized by judging the correlation degree between the calculation rules to avoid redundant calculation and redundant storage.

Description

technical field [0001] The present invention relates to a computing method and device for platformized log analysis, in particular to a computing method and device for avoiding redundant computing and redundant storage in platformized log analysis. Background technique [0002] Data platforms generally provide users with log analysis services, such as storing logs in HDFS, and users write Map / Reduce jobs and send them to the Hadoop platform for execution. In Stream Computing Storm, users also write similar tasks. Through such a platform service, each user only needs to spend a small learning cost to realize their own log analysis needs. However, a defect brought about by platformization is the waste of computing and storage resources. Two users may have very similar requirements but do not know the existence of the other party. Most of the calculation content is repeated; However, the entire database is used as the input object for traversal during calculation. These phenom...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F11/34
Inventor 储晓颖
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap