A cloud log collection method based on distributed architecture

A distributed architecture and acquisition method technology, applied in the information field, can solve the problem of inconsistency between the speed of log collection and processing, and achieve the effect of helping to analyze applications

Inactive Publication Date: 2019-02-15
INSPUR SOFTWARE CO LTD
View PDF8 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0011] A high-throughput distributed publish-subscribe message system that can store log streams with persistence and fault tolerance, and can solve the problem of inconsistency in the speed of log collection and processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A cloud log collection method based on distributed architecture

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] The content of the present invention is described in more detail below:

[0042] A cloud log collection method based on a distributed architecture of the present invention solves the problems of automatic collection, fast query and analysis of business system logs. It mainly includes the following log content:

[0043] 1) The system server's own log;

[0044] 2) Logs of applications deployed on the system server;

[0045] 3) Logs of third-party applications;

[0046] 4) System service log;

[0047] The present invention is implemented through the following schemes:

[0048] Environment deployment

[0049] According to the characteristics of the components, it is necessary to deploy Kafka clusters, Elasticsearch clusters, Logstash, install Filebeat components on each server, and finally build a log system framework, such as figure 1 shown

[0050] 1) Each server configures the log path to be collected by Filebeat, and sets the address of the transmission pipeline ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a cloud log collection method based on a distributed architecture, The invention belongs to the field of information technology. According to the characteristics of the components, the invention needs to deploy Kafka cluster, Elasticsearch cluster, Logstash, each server installs FileBeat component, and finally builds a log system framework. Realize the automatic collection and real-time query analysis of various logs and the fast import of the third-party logs.

Description

technical field [0001] The invention relates to the field of information technology, in particular to a cloud log collection method based on a distributed architecture, which solves the problem of log collection and query in the system. Background technique [0002] In an information system, the log plays a vital role. The log can record all the behaviors of the system. Through these behaviors, the usage of the system can be analyzed, and then the direction of optimization can be analyzed. In particular, abnormal information that occurs during system use can be used to remind relevant maintenance personnel to respond in time. [0003] In order to realize the automatic collection and real-time query and analysis of various logs of the system, as well as the rapid import of third-party logs, a log service system is required to complete this process. The following open-source components are analyzed as building blocks of the system. [0004] Elasticsearch [0005] As the sto...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/30G06F11/32
CPCG06F11/3006G06F11/3055G06F11/3072G06F11/321
Inventor 王玲
Owner INSPUR SOFTWARE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap