Distribution-base mass log collection system

A collection system and distributed technology, applied in the computer field, can solve the problems of refused connection, troublesome installation and deployment on the server side, etc., and achieve the effect of high reliability

Inactive Publication Date: 2014-09-10
蓝盾信息安全技术有限公司
View PDF4 Cites 132 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the installation and deployment of the server side in this method is more troublesome, and most of them need to be compiled and installed using source code. Moreover, open source software is basically foreign software. Therefore, it is possible that when downloading dependent software, there will be a problem of refusal of connection

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distribution-base mass log collection system
  • Distribution-base mass log collection system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0028] This paper proposes a distributed-based massive log collection system. By installing the Agent process on the target host, the text, application programs, database and other log information of the target host are selectively pushed to the unified access interface of the server cluster. The server side adopts distributed cache and real-time stream processing framework technology. The present invention solves the problem of using the traditional log collection...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distribution-base mass log collection system comprising a data source layer, a distribution-type cache layer, a distribution-type storing and calculating layer, a bushiness processing layer, a visible display layer and a unified dispatching and managing module. The system has the advantages that the problem of log collection and high-speed storage can be solved effectively, the distribution-type storage and search engine technology is adopted, querying and retrieval speed can be increased, and the mass logs can be collected and analyzed accurately and accurately in high speed.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a distributed-based massive log collection system. Background technique [0002] With the rise and development of cloud computing, mobile Internet, and Internet of Things, the era of big data has come. The log collection and analysis of the big data business system is the basis for evaluating the health of the node host, analyzing the problems in the system, and displaying various reports. [0003] There are many open source massive log solutions, such as Scribe of Facebook, Chukwa of Apache, Kafka of LinkedIn, Flume of Cloudera, etc. There are also comprehensive solutions, such as Kibana+Elasticsearch+LogStash, LogAnalyzer+MySQL+RSyslog, Splunk. These solutions all have three basic components, namely agent, collector, and store. The agent encapsulates the data source, sends the data in the data source to the collector, and the collector receives the data of multiple agents, s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F11/34
CPCG06F16/25
Inventor 杨育斌柯宗贵赵必厦
Owner 蓝盾信息安全技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap