Monitoring operation and maintenance alarm platform

Abstract

The invention relates to the field of monitoring operation and maintenance alarm, and provides a monitoring operation and maintenance alarm platform, which is characterized by comprising a data acquisition end, a management and control center and a centralized alarm center, wherein the data acquisition end supports container deployment, virtual machine deployment or physical machine deployment, the data acquisition end loads a TCP / IP (Transmission Control Protocol / Internet Protocol), and a binary message format and / or Kafka are / is used for supporting communication; when the monitoring task is received, the data acquisition end starts to acquire Agent data of the monitored end and sends the Agent data to the management and control center; the management and control center is used for storing and processing the Agent data sent by the data acquisition end and transmitting the processed Agent data to the centralized alarm center; and the centralized alarm center is used for giving an alarm according to the processed Agent data. According to the monitoring operation and maintenance platform provided by the invention, the data of the monitored end can be acquired in real time and analyzed, and when the analysis result represents abnormality, an alarm is given, so that important help is provided for identification and troubleshooting of operation and maintenance work.

Description

Technical Field

[0001] The present disclosure relates to the technical field of monitoring, operation and maintenance alarm, and in particular to a monitoring, operation and maintenance alarm platform. Background Art

[0002] At present, IT technology is developing rapidly, and the IT operating environment is becoming increasingly complex. Faced with the update and development of business and the gradual increase of operation and maintenance resources, the difficulty of operation and maintenance monitoring has increased. The current operation and maintenance monitoring tools on the market have a single monitoring direction, high operation and maintenance complexity and maintenance costs, and incomplete monitoring coverage. Therefore, an effective monitoring method that can cover business, environment, and equipment is needed to assist operation and maintenance work. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure provide a monitoring and operation and maintenance alarm platform to solve the problem of how to comprehensively and efficiently monitor the status and operation and maintenance of business, environment, and equipment in real time in the prior art.

[0004] According to a first aspect of an embodiment of the present disclosure, a monitoring and operation alarm platform is provided, characterized in that the monitoring and operation alarm platform comprises: a data collection terminal, a control center, and a centralized alarm center; wherein the data collection terminal supports container deployment, virtual machine deployment, or physical machine deployment, and supports operating system monitoring agent collection function, middleware monitoring agent collection function, network performance monitoring agent collection function, application performance monitoring agent collection function, and agent log collection function; the data collection terminal loads the TCP / IP protocol, and uses the binary message format and / or Kafka to support communication; when receiving a monitoring task, the data collection terminal starts to collect the Agent data of the monitored terminal, and sends the Agent data to the control center; the control center is used to store and process the Agent data sent by the data collection terminal, and transmit the processed Agent data to the centralized alarm center; the centralized alarm center is used to issue an alarm based on the processed Agent data.

[0005] In some optional implementations of some embodiments, the data acquisition terminal includes at least: an acquisition module, a reading module, a cache module, a sending module and a recording module; the acquisition module is used to acquire the path of the log file, the reading module is used to read the log file and transmit it to the cache module for caching, the cache module is used to cache the Agent data of the log file and has transmitted the log file to the sending module, and the sending module is used to send the log file to the back-end server; the recording module is used to record the log file and the corresponding offset.

[0006] In some optional implementation methods of some embodiments, there is a record file at the above-mentioned data acquisition end; when the above-mentioned cache module completes the log file caching, the file status of the above-mentioned log file is recorded in the above-mentioned record file; when the above-mentioned sending module sends the above-mentioned log file to the back-end server, the above-mentioned file status in the above-mentioned record file is updated after a confirmation event is generated.

[0007] In some optional implementations of some embodiments, the above-mentioned control center includes at least: a data storage module and a data processing module; the above-mentioned data storage module includes a time series database and an ES / ClickHouse database, wherein the above-mentioned time series database is used to store indicator data, and the storage time is a preset time period, and the above-mentioned ES / ClickHouse database is used to store the full amount of data and permanently store data; the above-mentioned data processing module has pre-set analysis and processing rules, supports basic environment and operation health analysis and processing functions for Agent data, supports business health data analysis and processing functions, and supports performance anomaly analysis functions.

[0008] In some optional implementations of some embodiments, the data processing module supports an external screen function and supports a function of receiving user operations to create analysis charts.

[0009] In some optional implementations of some embodiments, the above-mentioned management and control center has a reserved plug-in interface and database, which supports users to achieve docking through plug-ins and supports users to extend docking functions.

[0010] In some optional implementations of some embodiments, the centralized alarm center includes at least: a data processing alarm module; the data processing alarm module supports real-time processing of Agent data from one or more data sources, and supports alarming through preset alarm methods according to alarm rules.

[0011] In some optional implementations of some embodiments, the above-mentioned alarm rules are preset alarm rules and / or user-defined alarm rules; if they are user-defined alarm rules, the above-mentioned centralized alarm center adds the user-defined alarm rules to the database and activates them, and also supports the generation of user-defined tags.

[0012] In some optional implementations of some embodiments, if the centralized alarm center issues an alarm based on a user-defined alarm rule and there is a generated user-defined tag, the centralized alarm center will issue an alarm based on the generated user-defined tag.

[0013] In some optional implementations of some embodiments, the above-mentioned preset alarm methods include at least one or more of the following: WeChat alarm method, DingTalk alarm method, and SMS alarm method.

[0014] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: the monitoring and operation and maintenance platform provided by the present disclosure includes a data collection terminal, a control center, and a centralized alarm center. The data collection terminal can collect data from the monitored terminal in real time. The control center can store the collected data by type and demand, and can process and analyze the collected data using pre-set analysis and processing rules. The centralized alarm center can issue alarms based on the alarm rules according to the data transmitted by the control center, which provides important assistance for identification and troubleshooting of operation and maintenance work. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a schematic diagram of the structure of a monitoring, operation and maintenance alarm platform according to some embodiments of the present disclosure;

[0017] Figure 2 It is an architecture diagram of a monitoring, operation and maintenance alarm platform according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 Schematic diagram of the structure of the monitoring and operation alarm platform according to some embodiments of the present disclosure. Figure 1 As shown, the monitoring operation and maintenance alarm platform 101 includes: a data collection terminal 102, a control center 103, and a centralized alarm center 104; wherein the data collection terminal 102 supports container deployment, virtual machine deployment or physical machine deployment, and supports operating system monitoring agent collection function, middleware monitoring agent collection function, network performance monitoring agent collection function, application performance monitoring agent collection function, and agent log collection function. The data collection terminal 102 loads the TCP / IP protocol and uses the binary message format and / or Kafka to support communication; when receiving a monitoring task, the data collection terminal 102 starts to collect the agent data of the monitored terminal 105, and sends the agent data to the control center 103; the control center 103 is used to store and process the agent data sent by the data collection terminal 102 and transmit the processed agent data to the centralized alarm center 104; the centralized alarm center 104 is used to issue an alarm based on the processed agent data.

[0025] In some optional implementations of some embodiments, container deployment is performed through sidecar, and the logs of the business container are mounted in a read-only manner to the specified directory of the Agent. Then the Agent (collection end) reads the specified directory to parse the data and sends the logs to the corresponding topic of kafka (message middleware), and the storage format is stored in a key-value pair. The virtual machine deployment Agent (collection end) is deployed as a service, and the startup user must have read permission for the collection log directory.

[0026] It should be noted that the above-mentioned monitoring and operation and maintenance alarm platform 101 can be hardware or software. When the monitoring and operation and maintenance alarm platform 101 is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the monitoring and operation and maintenance alarm platform 101 is embodied as software (such as an electronic platform built with the help of physical devices), it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0027] It should be understood that Figure 1 The number of monitored terminals in FIG. 1 is only illustrative. Any number of monitored terminals may be provided according to implementation requirements.

[0028] Figure 2 FIG. 1 is an architecture diagram of a monitoring, operation and maintenance alarm platform according to some embodiments of the present disclosure. Figure 2 As shown, the monitoring operation and maintenance alarm platform 101 includes: a data collection terminal 102, a management and control center 103, and a centralized alarm center 104.

[0029] The above-mentioned data acquisition terminal 102 at least includes: an acquisition module 1021, a reading module 1022, a cache module 1023, a sending module 1024 and a recording module 1025; the above-mentioned acquisition module 1021 is used to collect the path of the log file, the above-mentioned reading module 1022 is used to read the above-mentioned log file and transmit it to the above-mentioned cache module for caching, the above-mentioned cache module 1023 is used to cache the Agent data of the above-mentioned log file and has transmitted the above-mentioned log file to the above-mentioned sending module, and the above-mentioned sending module 1024 is used to send the above-mentioned log file to the back-end server; the above-mentioned recording module 1025 is used to record the above-mentioned log file and the corresponding offset.

[0030] In some optional implementations of some embodiments, the data collection terminal 102 has a record file 1026; when the cache module 1023 completes the log file cache, the file status of the log file is recorded in the record file 1026; when the sending module 1024 sends the log file to the backend server, the file status in the record file 1026 is updated after a confirmation event is generated. In addition, when the file is renamed or the storage location is moved, each log file can be identified according to the inode information and device number information of the maintenance record file to ensure that the log will not be read repeatedly. Here, the maintenance record file is stored on the device of the Agent collection terminal. Each installation of a collection terminal will save a record in a binary file. The collection is based on the record search and update the record after completion.

[0031] The control center 103 at least includes: a data storage module 1031, a data processing module 1032; the data storage module 1031 includes a time series database 10311 and an ES / ClickHouse database 10312, wherein the time series database 10311 is used to store indicator data, and the storage time is a preset time period, and the ES / ClickHouse database 10312 is used to store full data and store data permanently. Here, the indicator data is not related data of the equipment, such as equipment status, equipment performance and other data. The preset processing time period is 1 year, which can also be adjusted according to actual needs. The time series database 10311 is mostly used to supply operation and maintenance related personnel, so that the operation and maintenance related personnel can query the overall or local operation status in real time. The ES / ClickHouse database 10312 is mostly used to supply R&D related personnel, such as query logs, fault analysis, etc., and is also used for real-time monitoring of abnormal logs to achieve monitoring purposes.

[0032] The above-mentioned data processing module 1032 has pre-set analysis and processing rules 10321, which supports the basic environment and operation health analysis and processing functions of Agent data (such as top indicators, avg indicators, sum indicators, group indicators) based on the basic environment real-time data, business real-time data, abnormal business real-time data, etc., and realizes real-time monitoring, display of system operation health status, and automatic inspection for real-time collection and analysis of the basic environment, so as to reduce manual operations, reduce human risks, and improve work efficiency; supports business health data analysis and processing functions, analyzes the business at each node, response, success rate, etc., discovers business anomalies and analyzes the business health status (which can be understood as the status of the business); supports performance anomaly analysis functions, monitors failed services based on a large amount of business logs, performance data, etc., so as to improve troubleshooting efficiency.

[0033] In some optional implementations of some embodiments, the data processing module 1032 supports an external screen function and supports receiving user operations to create analysis charts. When the external screen function is activated, the monitoring situation can be displayed and the display area can be customized. When the user creates a chart operation, the monitoring situation can be displayed in the form of a chart created by the user in response to the user's operation. Here, the data for creating a chart at least supports chart types, chart data, chart themes, etc. written by users using syntax such as Promsql, ES, and loki.

[0034] In some optional implementations of some embodiments, the control center has a reserved plug-in interface and database, which supports users to achieve docking through plug-ins and supports users to expand docking functions. Specifically, it can receive user-written data collection programs or alarm notification programs that have provided corresponding metrics. It can also be connected to the alarm notification program written by the customer through WEBhook, which can be achieved by simply entering the customer's WEBhook address and related information in the alarm rule.

[0035] The centralized alarm center 104 mentioned above at least includes: a data processing alarm module 1041; the above-mentioned data processing alarm module 1041 supports real-time processing of Agent (collection end) data of one or more data sources, the Agent (collection end) sends the log to the topic (topic) of kafka (message middleware), the alarm module 1041 uses the alarm consumption group for consumption processing (supports user-defined filtering rules), and the processed messages can be stored in the time series database (capable of fast query processing and trend display), the alarm module 1041 reads the corresponding rule data according to the rules, and supports alarming through the preset alarm method according to the alarm rules. The above-mentioned alarm rules are preset alarm rules and / or receive user-defined alarm rules; if they are user-defined alarm rules, the above-mentioned centralized alarm center 104 will add the user-defined alarm rules to the database and enable them, and also supports the generation of user-defined tags. Here, if the centralized alarm center issues an alarm based on a user-defined alarm rule and there is a generated user-defined tag, the centralized alarm center will issue an alarm based on the generated user-defined tag. The preset alarm method includes at least one or more of the following: WeChat alarm method, DingTalk alarm method, SMS alarm method.

[0036] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: the monitoring operation and maintenance platform provided by the present disclosure includes a data collection terminal, a control center, and a centralized alarm center. Among them, the data collection terminal can choose which method to deploy and which monitoring function to start according to actual business needs, and can collect data from the monitored terminal in real time, complete a series of tasks such as collection, reading, caching, sending and recording, and there is a record file. After the cache module is completed, the file status of the log file will be recorded in the record file. When the log file is sent, a confirmation event is generated to update the file status in the record file. If the file is renamed or the storage location is moved, each log file can be identified according to the inode information and device number information of the maintenance record file to ensure that the log will not be read repeatedly. The control center can store the collected data by type and demand, and can also use the pre-set analysis and processing rules to process the collected data, basic environment analysis, business real-time data analysis, and abnormal business real-time data analysis, so as to understand the system operation, business monitoring and fault conditions. In addition, the control center supports external screen functions and auxiliary tabulation functions based on user operations. There are also reserved plug-in interfaces and databases so that users can connect their own programs to this monitoring and operation platform. The centralized alarm center can issue alarms based on the alarm rules according to the data transmitted by the control center, which provides important help for the identification and troubleshooting of operation and maintenance work. In addition to pre-setting, alarm rules can also be added according to user needs, making alarms more accurate. Supporting the generation of user-defined tags also improves the user experience of the alarm function.

[0037] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0038] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0039] The units described in some embodiments of the present disclosure may be implemented by software or hardware, and the described units may also be set in a processor.

[0040] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0041] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A monitoring and operation alarm platform, characterized in that: The monitoring operation and maintenance platform includes: Data collection terminal, control center, centralized alarm center; The data collection end supports container deployment, virtual machine deployment or physical machine deployment, supports operating system monitoring agent collection function, middleware monitoring agent collection function, network performance monitoring agent collection function, application performance monitoring agent collection function, agent log collection function, and loads TCP / IP protocol and uses binary message format and / or Kafka to support communication; When receiving a monitoring task, the data collection end starts to collect Agent data of the monitored end, and sends the Agent data to the control center; The control center is used to store and process the Agent data sent by the data acquisition terminal and transmit the processed Agent data to the centralized alarm center; The centralized alarm center is used to issue an alarm based on the processed Agent data.

2. The monitoring and operation alarm platform according to claim 1 is characterized in that: The data acquisition terminal at least includes: an acquisition module, a reading module, a buffer module, a sending module and a recording module; The acquisition module is used to collect the path of the log file. The reading module is used to read the log file and transmit it to the cache module for caching. The cache module is used to cache the Agent data of the log file and has transmitted the log file to the sending module. The sending module is used to send the log file to the back-end server; The recording module is used to record the log file and the corresponding offset.

3. The monitoring, operation and maintenance alarm platform according to claim 2 is characterized in that: The data collection end has a record file; When the cache module completes the log file cache, the file status of the log file is recorded in the record file; When the sending module sends the log file to the backend server, a confirmation event is generated and the file status in the log file is updated.

4. The monitoring, operation and maintenance alarm platform according to claim 1, characterized in that: The control center at least includes: a data storage module and a data processing module; the data storage module includes a time series database and an ES / ClickHouse database, wherein the time series database is used to store indicator data for a preset time period, and the ES / ClickHouse database is used to store the full amount of data and permanently store data; the data processing module has pre-set analysis and processing rules, supports basic environment and operation health status analysis and processing functions for Agent data, supports business health status data analysis and processing functions, and supports performance anomaly analysis functions.

5. The monitoring, operation and maintenance alarm platform according to claim 4 is characterized in that: The data processing module supports an external screen function and supports a function of receiving user operations to create analysis charts.

6. The monitoring operation and maintenance alarm platform according to claim 1, characterized in that: The control center has a reserved plug-in interface and database, which supports users to achieve docking through plug-ins and supports users to expand docking functions.

7. The monitoring, operation and maintenance alarm platform according to claim 1, characterized in that: The centralized alarm center at least includes: a data processing alarm module; the data processing alarm module supports real-time processing of Agent data from one or more data sources, and supports alarming in a preset alarm manner according to alarm rules.

8. The monitoring, operation and maintenance alarm platform according to claim 7, characterized in that: The alarm rules are preset alarm rules and / or user-defined alarm rules; if they are user-defined alarm rules, the centralized alarm center adds the user-defined alarm rules to the database and activates them, and also supports the generation of user-defined tags.

9. The monitoring operation and maintenance alarm platform according to claim 8, characterized in that: If the centralized alarm center issues an alarm according to a user-defined alarm rule and there is a generated user-defined tag, the centralized alarm center will issue an alarm based on the generated user-defined tag.

10. The monitoring, operation and maintenance alarm platform according to claim 7, characterized in that: The preset alarm method includes at least one or more of the following: WeChat alarm method, DingTalk alarm method, and SMS alarm method.

Images