Distributed monitoring system based on hadoop cluster and method thereof

A Hadoop cluster and distributed monitoring technology, applied in transmission systems, digital transmission systems, electrical components, etc., can solve problems such as single-point obstacles, network utilization bottlenecks, etc., to reduce utilization, enhance tuning experience, and facilitate data obtained effect

Inactive Publication Date: 2014-03-05
SUGON INFORMATION IND
View PDF5 Cites 37 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Ganglia will have problems such as single point obstacles and network utilization bottlenecks in large-scale cluster monitoring solutions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed monitoring system based on hadoop cluster and method thereof
  • Distributed monitoring system based on hadoop cluster and method thereof
  • Distributed monitoring system based on hadoop cluster and method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0060] Such as figure 1 As shown, the present invention includes a collection agent module, a resource update module, a network monitoring module, a collection scheduling module, an alarm generation module and a hadoop cluster monitoring module.

[0061] Among them, the collection agent module, the network monitoring module, and the collection scheduling module jointly complete the physical resource collection and performance monitoring of each node of the cluster.

[0062] Among them, the Hadoop cluster monitoring module completes the monitoring of the indicators of each component of Hadoop through the design of JMX indicators.

[0063] Wherein, the alarm generating module analyzes all collected data to generate alarm information.

[0064] Resources: generally refers to entities that contain index attributes, such as ph...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a distributed monitoring system based on a hadoop cluster and a method thereof. The system comprises a collection agency module, a resource updating module, a network monitoring module, a collection scheduling module, a warning generating module and a hadoop cluster monitoring module, wherein the hadoop cluster monitoring module is connected with the collection agency module, the resource updating module, the network monitoring module, the collection scheduling module and the warning generating module. The method includes the steps of (1) collecting needed detection resources and all index information, (2) storing and caching the collected data, (3) scheduling and updating collected information, (4) timely sampling to generate a data graph needed by a foreground web, (5) monitoring the network state of nodes in the cluster, and (6) determining whether a warning needs to be sent out according to a threshold value and generating warning information if the warning is needed. The distributed monitoring system can serve as an independent cluster monitoring system, a basic index database can be conveniently extended, a current index value can be taken back, a convergence view of indexes can be checked, and the trend of data timing sequences can be observed.

Description

technical field [0001] The invention belongs to monitoring technology, and specifically relates to a hadoop cluster-based distributed monitoring system and method thereof. Background technique [0002] Hadoop is an open source distributed system infrastructure developed by the Apache Foundation. Users can develop distributed programs without knowing the underlying details of the distribution. Make full use of the power of the cluster for high-speed computing and storage. Hadoop implements a distributed file system (Hadoop Distributed File System), referred to as HDFS. HDFS has the characteristics of high fault tolerance and is designed to be deployed on low-cost (low-cost) hardware. And it provides high throughput (high throughput) to access application data, suitable for those applications with large data sets (large data set). The Hadoop framework has the characteristics of high reliability, high scalability, high efficiency, and high fault tolerance. [0003] With th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/26H04L29/06
Inventor 耿庆民郭庆谢莹莹苗艳超刘新春邵宗有
Owner SUGON INFORMATION IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap