Log analysis system and method based on distributed collection

An analysis system and distributed technology, applied in the field of data processing, can solve problems such as log management and correlation analysis

Active Publication Date: 2017-01-25
GUANGDONG ESHORE TECH
View PDF3 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Due to the large amount of transaction data in the telecommunication system and the large number of application services, and most of the application services are deployed in clusters, a large number of logs ar

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log analysis system and method based on distributed collection

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0024] Such as figure 1 Shown is the system architecture diagram of the present invention. A log analysis system based on distributed collection of the present invention includes a log file collection application module, a log receiving application cluster module, a distributed log storage module and a Zookeeper registration center cluster module.

[0025] The log file collection application module collects and monitors the component log files of the current machi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a log analysis system and method based on distributed collection. The system comprises a log file collecting application module, a log receiving application cluster module, a distributed type log storage module and a Zookeeper registration center cluster module. In addition, the invention further discloses a log analysis method based on distributed collection. By means of the system and method, the horizontal expansion capacity of a log processing layer and a log storage layer is improved, cross-service calling logs are grouped by generating request identification (Tracing ID) for each request according to a thread isolation principle, and data log circulation and abnormity details in the service calling process are clearly positioned.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a log analysis system and method based on distributed collection. Background technique [0002] Due to the large volume of transaction data in the telecommunications system, many application services, and most of the application services are deployed in clusters, a large number of logs are generated every day, which are generally streaming data, such as search engine pv, queries, etc. These logs are stored independently of each other and cannot Perform unified management and correlation analysis on these logs. How to understand the system status in time by analyzing them, discover system abnormalities, and respond quickly to abnormal events at the first time is an urgent problem to be solved in network and system management. Contents of the invention [0003] The purpose of the present invention is to overcome the defects of the prior art and provide a log analysis sys...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/1815
Inventor 周宏江廖小文周恒马俊贤
Owner GUANGDONG ESHORE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap