Monitoring data real-time storage method and system, electronic equipment and storage medium

A technology for monitoring data and real-time, applied in the field of big data, can solve the problem of low data query efficiency, and achieve the effect of improving data reading ability, facilitating retrieval and query, and improving data reading performance

Active Publication Date: 2020-05-12
WUHAN OPTICS VALLEY INFORMATION TECH
View PDF9 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of this, the embodiment of the present invention provides a monitoring data real-time storage method, system, electronic equipment and storage medium to solve the problem of low data query efficiency in existing large-scale data storage methods

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Monitoring data real-time storage method and system, electronic equipment and storage medium
  • Monitoring data real-time storage method and system, electronic equipment and storage medium
  • Monitoring data real-time storage method and system, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] In order to make the purpose, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the following The described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work belong to the protection scope of the present invention. The principles and features of the present invention will be described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention, not to limit the scope of the present invention.

[0019] The term "comprising" and other expressions with similar meanings in the description or claims of the p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a monitoring data real-time storage method and a system, electronic equipment and a storage medium. The method comprises the following steps: acquiring monitoring data of monitoring equipment through Zabbix; pushing the monitoring data to a Kafka monitoring item queue in real time, acquiring monitoring item queue data by Spark Streaming, carrying out stream processing, and storing the monitoring item queue data to an HBase predetermined data partition, the data partition being a storage table segmented according to different durations set according to a scheduling period. By means of the method, the problems that large-scale monitoring data query and analysis efficiency is low and large-scale monitoring data storage is written into hotspots are solved, large-batch data storage is achieved based on the HBase, meanwhile, different data partitions are set to improve data query efficiency, and data analysis and utilization are facilitated.

Description

technical field [0001] The invention relates to the field of big data, in particular to a monitoring data real-time storage method, system, electronic equipment and storage medium. Background technique [0002] Zabbix is ​​a monitoring tool with distributed system monitoring and network monitoring functions based on the web interface. It can monitor the status of the server's CPU, memory, network, and hard disk lights. A monitoring device (such as a server) has multiple monitoring indicators, and data is collected for each monitoring indicator in a fixed period, so the amount of data generated for Zabbix monitoring is very large, which undoubtedly puts forward higher requirements for data storage and analysis. [0003] Usually, MySQL is used to store monitoring data, which is difficult to meet the requirements of large-scale data storage and analysis. However, time-series databases such as OpenTSDB can meet the requirements of large-scale data storage, but the structure of t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/30G06F16/22G06F16/2458
CPCG06F11/3006G06F11/3065G06F16/2282G06F16/2462Y02D10/00
Inventor 谢邵虎朱明杰
Owner WUHAN OPTICS VALLEY INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap