Distributed log management method and system

A management method and technology of a management system, applied in the field of distributed log management methods and systems, can solve problems such as time-consuming and laborious, difficult to locate, and difficult for stand-alone applications to support large-scale user access, so as to reduce coupling and realize distributed management , Realize the effect of visual analysis

Active Publication Date: 2019-11-05
INSPUR COMMON SOFTWARE
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Viewing system logs is an important way to analyze and solve online problems. Usually, we output the logs to the console or local files. When troubleshooting, we search the local logs according to keywords. This is very convenient and effective in stand-alone applications. However, with the expansion of user scale, it is difficult for stand-alone applications to support large-scale user access services. At this time, distributed architecture is adopted in project development, and logs will be recorded in multiple servers or files. The same service usually adopts cluster Deployment, it is difficult to locate a specific machine for an exception log, and analyzing the problem may require log query for each cluster machine
Doing so is obviously time-consuming and labor-intensive

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed log management method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] A distributed log management method of the present invention is used for collecting and managing log files distributed in each independent service system.

[0034] The log management method includes the following steps:

[0035] S100. Collect log files of each independent service system based on a unified requestId, and connect the distributed log files in series;

[0036] S200. Store the log file in a document-oriented database, and build an index cluster in the database, where each index in the index cluster corresponds to a log file;

[0037] S300. Interact with the user through a web page, and the web page cooperates with the database through a graphic interface to realize data query and data display.

[0038] Among them, each independent service system is configured on the corresponding server, and the log file corresponding to the service system is collected through the AOP component, and each log file is filtered to obtain a log file in a specific format, and al...

Embodiment 2

[0042] A distributed log management system of the present invention is used to collect and manage log files distributed in each independent service system, and the log management system includes a log collection module, a storage search module and a query display module.

[0043] There are several log collection modules that correspond to the independent service systems one by one. Each log collection module is configured on the server of the corresponding independent service system, and the log files of the corresponding independent service system are collected based on the unified requestId through the AOP component.

[0044] In this embodiment, the log collection module includes an input pipeline, a filter, and an output pipeline. Both the input pipeline and the output pipeline are used as data flow channels. After collecting the log files of the corresponding independent service system through the AOP component based on a unified requestId, the The pipeline transmits the lo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed log management method and a distributed log management system, belongs to log file management, and aims to solve the technical problem of how to realize log management and log query in a distributed manner. The method comprises the following steps: collecting log files of each independent service system based on a unified request Id, and connecting the distributed log files in series; storing the log files in a document-oriented database, and constructing an index cluster in the database, wherein each index in the index cluster corresponds to one log file;interacting with a user through a web page, and matching the web page with a database through an image interface to realize data query and data display. The system comprises a log acquisition module,a storage search module and a query display module.

Description

technical field [0001] The invention relates to the field of log file management, in particular to a distributed log management method and system. Background technique [0002] Viewing system logs is an important way to analyze and solve online problems. Usually, we output the logs to the console or local files. When troubleshooting, we search the local logs according to keywords. This is very convenient and effective in stand-alone applications. However, with the expansion of user scale, it is difficult for stand-alone applications to support large-scale user access services. At this time, distributed architecture is adopted in project development, and logs will be recorded in multiple servers or files. The same service usually adopts cluster For deployment, even an exception log is difficult to locate a specific machine. To analyze the problem, each cluster machine may need to perform log query. Doing so is obviously time-consuming and labor-intensive. At this time, a di...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/18G06F16/13
CPCG06F16/1815G06F16/134
Inventor 王超邵辉王宏达
Owner INSPUR COMMON SOFTWARE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap