Log-based big data analysis method

An analysis method and big data technology, applied in the field of log files, can solve the problems of log statistics and retrieval troubles, inaccurate data, low efficiency, etc., and achieve convenient and intuitive indexing and searching
CN108804497AInactive Publication Date: 2018-11-13BEIJING CHINA POWER INFORMATION TECH +3
3 Cites 8 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
BEIJING CHINA POWER INFORMATION TECH
Publication Date
2018-11-13
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a log-based big data analysis method, the log-based big data analysis method mainly includes the steps of collecting data in each server according to a configuration file of arelated Beats application; and collecting the data collected by the Beats application by LogStash and storing the data in ElasticSearch in an order generated by automatic indexing. The ElasticSearch provides a concise Restful-style query interface to the outside world and develops Lucene through the Java program and uses the Lucene as its core to realize query data calling, i.e. all indexing and searching functions are realized, the functions of accurately locating and searching log information and analyzing related data are achieved, and accurate business data is acquired. Finally, Echart isused for visual display to hide the complexity of the Lucene, which makes indexing and searching easy and intuitive.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of log files, in particular to a log-based big data analysis method. Background technique

[0002] In recent years, due to the continuous growth of data in the Internet field, companies are faced with the need to process massive data. Data analysis mainly provides support for the company's operation and maintenance decision-making, mainly based on the logs generated by each server. Based on the data information provided by these logs, the user access and data flow are of great significance in terms of time dimension, product line dimension, and domain name dimension. Clear quantitative data, so as to achieve the purpose of providing reference for the company's server operation and maintenance management, traffic distribution, and forecasting.

[0003] The main technology of the existing scheme is to adopt the ELK-Stack (elasticsearch, logstash&kibana, analysis and visualization platform) framework, which is...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More