Intelligent operation and maintenance management system

Abstract

The invention discloses an intelligent operation and maintenance management system, and relates to the technical field of system operation and maintenance management, the intelligent operation and maintenance management system comprises a data acquisition module, a data processing module and a data analysis module, the data acquisition module collects performance data from a network device, a Windows server and an application program by using a simple network management protocol, a WMI interface and an API interface, and the data processing module processes the performance data and analyzes the performance data. The data processing module is used for establishing a unified technical monitoring standard and realizing comprehensive real-time monitoring and data analysis of a technical environment by utilizing an autoencoder anomaly detection algorithm, and rapidly finding and responding to anomaly and fault, so that the technical environment is monitored and monitored in real time; and the data analysis module is used for setting an intelligent alarm and early warning rule based on an abnormal detection result, immediately sending out early warning information once a problem is detected, and presenting an analysis result in a graphical mode, so that operation and maintenance personnel are assisted to intuitively understand a system state and an operation and maintenance effect, and efficient operation and maintenance work is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of system operation and maintenance management, and in particular to an intelligent operation and maintenance management system. Background Art

[0002] The current development prospects of intelligent operation and maintenance management systems are very broad, not only because the enterprise IT architecture is becoming increasingly complex, but also because new technologies such as cloud computing, big data, and the Internet of Things are changing our working environment at an unprecedented speed. The traditional operation and maintenance model, which relies on manual monitoring and manual processing, has been unable to adapt to this efficient, stable, and real-time operation and maintenance needs. The emergence of intelligent operation and maintenance management systems is precisely to meet this challenge. It uses advanced technical means to achieve comprehensive monitoring, intelligent analysis, and early warning of the IT environment, thereby greatly improving the efficiency and accuracy of operation and maintenance.

[0003] However, with the continuous emergence of new technology environments, intelligent operation and maintenance management systems are also facing new challenges. New technologies such as containerization, microservice architecture, and Serverless have brought new complexity and uncertainty to operation and maintenance. Traditional monitoring methods often find it difficult to cover these new technology environments, resulting in insufficient controllability and sensitivity of supervision. In order to solve this problem, it is necessary to formulate unified new technology monitoring standards and specifications to ensure the comprehensiveness and effectiveness of monitoring. At the same time, by introducing anomaly detection algorithms based on machine learning, historical data can be analyzed and learned, and anomaly detection models can be built to accurately identify unknown anomalies in new technology environments, thereby further improving the intelligence level of operation and maintenance. In addition, intelligent operation and maintenance management systems also need to set intelligent alarm and warning rules. These rules are customized according to business needs and system characteristics, and reasonable alarm thresholds and warning conditions are set to ensure timely detection and response when anomalies and failures occur. In summary, intelligent operation and maintenance management systems still face many challenges in their development, but as long as they keep up with the pace of technological development, they will be able to effectively solve these problems and promote the development of intelligent operation and maintenance management systems to a higher level. In the future, intelligent operation and maintenance management systems will become an important support for the digital transformation of enterprises, providing strong guarantees for the business development and innovation of enterprises. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides an intelligent operation and maintenance management system that solves the problems in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: an intelligent operation and maintenance management system, including an operation and maintenance data acquisition module, an operation and maintenance data processing module and an operation and maintenance data analysis module; The operation and maintenance data collection module collects performance data from network devices, Windows servers and applications through the simple network management protocol, WMI and API interface respectively, collects and analyzes log data centrally through the Splunk log collection tool, and sends event notifications to the operation and maintenance data collection module through the event trigger mechanism from the target device, server and application to collect event data; The operation and maintenance data processing module implements comprehensive monitoring and real-time data analysis of the technical environment by formulating unified technical monitoring standards and specifications and adopting an anomaly detection algorithm based on an autoencoder; The operation and maintenance data analysis module sets intelligent alarm and warning rules according to the analysis results of the anomaly detection model of the autoencoder. When anomalies and failures are detected, warning information is issued in a timely manner and the analysis results are displayed in a graphical manner.

[0006] Furthermore, the process of the operation and maintenance data collection module collecting performance data from network devices, Windows servers and applications through the simple network management protocol, WMI and API interface includes: The operation and maintenance data collection module sends a request to the network device through the SNMP protocol. The network device responds to the request and returns its performance data, which includes CPU usage, memory occupancy and interface traffic. The operation and maintenance data collection module periodically sends requests to obtain the latest performance data and stores it in the database. It uses the WMI tool for managing the Windows system to collect performance data from the Windows server, queries performance counters through WMI, obtains the server's CPU, memory and disk performance indicators, and regularly executes WMI queries to obtain the latest data. At the same time, the operation and maintenance data collection module obtains the performance data of the application by periodically calling the API interface provided by the application, and stores the returned data in the database.

[0007] Furthermore, the operation and maintenance data collection module uses the Splunk log collection tool to centrally collect and analyze log data, including: The operation and maintenance data collection module uses Splunk's log collection function to centrally collect log data from network devices and systems, parses different log formats through Splunk, and converts them into structured data formats. The operation and maintenance data collection module stores the parsed log data in the database for subsequent analysis and query.

[0008] Furthermore, the operation and maintenance data collection module sends an event notification to the operation and maintenance data collection module through an event triggering mechanism, and the process of collecting event data includes: When specific events occur in the target device, server and application, event notifications are sent to the data acquisition module based on preset rules. The specific events include system anomalies, performance bottlenecks and user operations. After receiving the notification, the operation and maintenance data acquisition module immediately starts the collection process, parses and processes the data in the event notification, and extracts key event information. The information is then stored in the database for subsequent analysis, monitoring and alarm operations.

[0009] Furthermore, the operation and maintenance data processing module formulates a unified technical monitoring standard and adopts an anomaly detection algorithm based on an autoencoder to achieve a comprehensive monitoring of the technical environment and a real-time data analysis process, including: The operation and maintenance data processing module formulates a unified technical monitoring standard to make the data collected from network devices, Windows servers and applications consistent and comparable. The unified technical monitoring standard includes the format, content, collection frequency and storage method of the data, defines the data type, data format, collection time interval and data transmission protocol provided by various data sources, so that all monitored objects output data according to unified specifications, requiring the data to be complete and without missing, the timestamp to be accurate to the second, and the data delay to be no more than 5 minutes. At the same time, a data verification mechanism is established to mark and process abnormal and unreasonable data, formulate data cleaning rules, deduplicate, denoise and format convert the collected raw data, establish a data integration process, clean, integrate and normalize the data from different data sources to form a unified data view, and build an anomaly detection model based on the autoencoder. The input layer of the autoencoder receives the data collected from the operation and maintenance data. The raw data transmitted by the collection module includes performance data, log data and event data of network devices, Windows servers and applications. The input layer passes the data to the next layer for processing. The neurons in the hidden layer of the autoencoder are responsible for extracting the feature representation of the input data, mapping the input data to a low-dimensional coding space through nonlinear transformation, forming a coded representation of the data, and enabling the autoencoder to learn the normal feature mode of the data. The output layer of the autoencoder receives the coded representation transmitted by the hidden layer, and reconstructs it into an approximate value of the input data through decoding. The reconstruction error is calculated by comparing the difference between the reconstructed data and the original input data. If the reconstruction error exceeds the preset error threshold, it indicates that there is an abnormality in the data. If the reconstruction error is within the preset error threshold, it indicates that the data is normal. At the same time, the output layer passes the reconstructed data and the reconstruction error to the data analysis module for the next step of judgment and processing.

[0010] Furthermore, the operation and maintenance data processing module sets intelligent alarm and warning rules according to the analysis results of the anomaly detection model of the autoencoder. When anomalies and failures are detected, the process of issuing warning information in a timely manner includes: When new data is input into the trained autoencoder, if the input data is normal, the output layer reconstructs a result similar to the input data, which is judged as normal data and does not trigger an early warning. If the input data contains anomalies, the reconstruction error increases significantly and is judged as abnormal data, thereby triggering the anomaly detection mechanism. The operation and maintenance data analysis module immediately generates an early warning message, which includes the type, level, occurrence time and information of related equipment and applications. The early warning message is sent to relevant operation and maintenance personnel by email.

[0011] Furthermore, the process of displaying the analysis results in a graphical manner by the operation and maintenance data analysis module includes: For the output results of the autoencoder anomaly detection model, a line chart is used to show the changing trend of the data. When the reconstruction error increases significantly, it is highlighted with different marks. At the same time, an interactive function is provided to allow operation and maintenance personnel to obtain detailed information by clicking on the data points in the chart.

[0012] The present invention provides an intelligent operation and maintenance management system. It has the following beneficial effects: First, through the efficient operation of the data acquisition module, the system can comprehensively and real-time collect the performance data, log data and event data of network devices, Windows servers and applications, ensuring that the operation and maintenance personnel can obtain the most comprehensive system status information. Secondly, the data processing module adopts the autoencoder anomaly detection algorithm to formulate a unified technical monitoring standard to achieve comprehensive real-time monitoring and data analysis of the technical environment. This intelligent monitoring method can quickly discover and respond to anomalies and faults in the system, greatly improving the efficiency and accuracy of operation and maintenance work. Finally, the data analysis module sets intelligent alarm and warning rules based on the anomaly detection results. Once a problem is detected, the system immediately issues a notification and presents the analysis results in a graphical way. This intuitive and easy-to-understand presentation method can not only help operation and maintenance personnel quickly locate problems, but also provide them with effective decision-making support, thereby ensuring the efficient operation and maintenance work. In summary, the intelligent operation and maintenance management system can significantly improve the efficiency and quality of operation and maintenance work, reduce operation and maintenance costs, and provide strong technical support for the stable development of enterprises. At the same time, the intelligent and automated characteristics of the system can also adapt to the increasingly complex operation and maintenance environment, laying a solid foundation for the future development of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram of an intelligent operation and maintenance management system according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] like Figure 1 As shown, the present invention provides a technical solution: an intelligent operation and maintenance management system, including an operation and maintenance data acquisition module, an operation and maintenance data processing module and an operation and maintenance data analysis module; The operation and maintenance data collection module collects performance data from network devices, Windows servers and applications through the simple network management protocol, WMI and API interface respectively, collects and analyzes log data centrally through the Splunk log collection tool, and sends event notifications to the operation and maintenance data collection module through the event trigger mechanism from the target device, server and application to collect event data; The operation and maintenance data processing module implements comprehensive monitoring and real-time data analysis of the technical environment by formulating unified technical monitoring standards and specifications and adopting an anomaly detection algorithm based on an autoencoder; The operation and maintenance data analysis module sets intelligent alarm and warning rules according to the analysis results of the anomaly detection model of the autoencoder. When anomalies and failures are detected, warning information is issued in a timely manner and the analysis results are displayed in a graphical manner.

[0016] The process of the operation and maintenance data collection module collecting performance data from network devices, Windows servers and applications through the simple network management protocol, WMI and API interface includes: The operation and maintenance data collection module sends a request to the network device through the SNMP protocol. The network device responds to the request and returns its performance data, which includes CPU usage, memory occupancy and interface traffic. The operation and maintenance data collection module periodically sends requests to obtain the latest performance data and stores it in the database. It uses the WMI tool for managing the Windows system to collect performance data from the Windows server, queries performance counters through WMI, obtains the server's CPU, memory and disk performance indicators, and regularly executes WMI queries to obtain the latest data. At the same time, the operation and maintenance data collection module obtains the performance data of the application by periodically calling the API interface provided by the application, and stores the returned data in the database.

[0017] The operation and maintenance data collection module uses the Splunk log collection tool to centrally collect and analyze log data, including: The operation and maintenance data collection module uses Splunk's log collection function to centrally collect log data from network devices and systems, parses different log formats through Splunk, and converts them into structured data formats. The operation and maintenance data collection module stores the parsed log data in the database for subsequent analysis and query.

[0018] The operation and maintenance data collection module sends event notifications to the operation and maintenance data collection module through an event triggering mechanism, and the process of collecting event data includes: When specific events occur in the target device, server and application, event notifications are sent to the data acquisition module based on preset rules. The specific events include system anomalies, performance bottlenecks and user operations. After receiving the notification, the operation and maintenance data acquisition module immediately starts the collection process, parses and processes the data in the event notification, and extracts key event information. The information is then stored in the database for subsequent analysis, monitoring and alarm operations.

[0019] The operation and maintenance data processing module formulates a unified technical monitoring standard and adopts an anomaly detection algorithm based on an autoencoder to achieve comprehensive monitoring of the technical environment and real-time data analysis. The process includes: The operation and maintenance data processing module formulates a unified technical monitoring standard to make the data collected from network devices, Windows servers and applications consistent and comparable. The unified technical monitoring standard includes the format, content, collection frequency and storage method of the data, defines the data type, data format, collection time interval and data transmission protocol provided by various data sources, so that all monitored objects output data according to unified specifications, requiring the data to be complete and without missing, the timestamp to be accurate to the second, and the data delay to be no more than 5 minutes. At the same time, a data verification mechanism is established to mark and process abnormal and unreasonable data, formulate data cleaning rules, deduplicate, denoise and format convert the collected raw data, establish a data integration process, clean, integrate and normalize the data from different data sources to form a unified data view, and build an anomaly detection model based on the autoencoder. The input layer of the autoencoder receives the data collected from the operation and maintenance data. The raw data transmitted by the collection module includes performance data, log data and event data of network devices, Windows servers and applications. The input layer passes the data to the next layer for processing. The neurons in the hidden layer of the autoencoder are responsible for extracting the feature representation of the input data, mapping the input data to a low-dimensional coding space through nonlinear transformation, forming a coded representation of the data, and enabling the autoencoder to learn the normal feature mode of the data. The output layer of the autoencoder receives the coded representation transmitted by the hidden layer, and reconstructs it into an approximate value of the input data through decoding. The reconstruction error is calculated by comparing the difference between the reconstructed data and the original input data. If the reconstruction error exceeds the preset error threshold, it indicates that there is an abnormality in the data. If the reconstruction error is within the preset error threshold, it indicates that the data is normal. At the same time, the output layer passes the reconstructed data and the reconstruction error to the data analysis module for the next step of judgment and processing.

[0020] The operation and maintenance data processing module sets intelligent alarm and warning rules according to the analysis results of the anomaly detection model of the autoencoder. When anomalies and failures are detected, the process of issuing warning information in a timely manner includes: When new data is input into the trained autoencoder, if the input data is normal, the output layer reconstructs a result similar to the input data, which is judged as normal data and does not trigger an early warning. If the input data contains anomalies, the reconstruction error increases significantly and is judged as abnormal data, thereby triggering the anomaly detection mechanism. The operation and maintenance data analysis module immediately generates an early warning message, which includes the type, level, occurrence time and information of related equipment and applications. The early warning message is sent to relevant operation and maintenance personnel by email.

[0021] The process of displaying the analysis results in a graphical manner by the operation and maintenance data analysis module includes: For the output results of the autoencoder anomaly detection model, a line chart is used to show the changing trend of the data. When the reconstruction error increases significantly, it is highlighted with different marks. At the same time, an interactive function is provided to allow operation and maintenance personnel to obtain detailed information by clicking on the data points in the chart.

[0022] First, the user logs in to the system and enters the user name and password. The first login requires contacting the administrator to register and activate the account. After logging in, the system displays a list of devices, including the device name, model, status and manufacturer. Secondly, the user uses the data collection module to collect performance data from network devices, Windows servers and applications, and uses the Splunk tool to collect log data. At the same time, the system receives event notifications from the target device to collect event data. Then, the data processing module uses the autoencoder anomaly detection algorithm according to unified technical monitoring standards to conduct comprehensive real-time monitoring and analysis of the collected data, and quickly discover and respond to anomalies and faults. Finally, the data analysis module sets intelligent alarm and warning rules based on the anomaly detection results. Once a problem is detected, the system immediately issues a notification and presents the analysis results in a graphical way to help operation and maintenance personnel intuitively understand the system status and operation and maintenance results, thereby ensuring the efficient progress of operation and maintenance work.

[0023] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions. The sentence "includes an element defined by ... does not exclude the existence of other identical elements in the process, method, article or device including the element".

[0024] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent operation and maintenance management system, comprising an operation and maintenance data acquisition module, an operation and maintenance data processing module and an operation and maintenance data analysis module; The operation and maintenance data collection module collects performance data from network devices, Windows servers and applications through the simple network management protocol, WMI and API interface respectively, collects and analyzes log data centrally through the Splunk log collection tool, and sends event notifications to the operation and maintenance data collection module through the event trigger mechanism from the target device, server and application to collect event data; The operation and maintenance data processing module implements comprehensive monitoring and real-time data analysis of the technical environment by formulating unified technical monitoring standards and specifications and adopting an anomaly detection algorithm based on an autoencoder; The operation and maintenance data analysis module sets intelligent alarm and warning rules according to the analysis results of the anomaly detection model of the autoencoder. When anomalies and failures are detected, warning information is issued in a timely manner and the analysis results are displayed in a graphical manner.

2. The intelligent operation and maintenance management system according to claim 1, characterized in that: The process of the operation and maintenance data collection module collecting performance data from network devices, Windows servers and applications through the simple network management protocol, WMI and API interface includes: The operation and maintenance data collection module sends a request to the network device through the SNMP protocol. The network device responds to the request and returns its performance data, which includes CPU usage, memory occupancy and interface traffic. The operation and maintenance data collection module periodically sends requests to obtain the latest performance data and stores it in the database. It uses the WMI tool for managing the Windows system to collect performance data from the Windows server, queries performance counters through WMI, obtains the server's CPU, memory and disk performance indicators, and regularly executes WMI queries to obtain the latest data. At the same time, the operation and maintenance data collection module obtains the performance data of the application by periodically calling the API interface provided by the application, and stores the returned data in the database.

3. The intelligent operation and maintenance management system according to claim 2, characterized in that: The operation and maintenance data collection module uses the Splunk log collection tool to centrally collect and analyze log data, including: The operation and maintenance data collection module uses Splunk's log collection function to centrally collect log data from network devices and systems, parses different log formats through Splunk, and converts them into structured data formats. The operation and maintenance data collection module stores the parsed log data in the database for subsequent analysis and query.

4. The intelligent operation and maintenance management system according to claim 3, characterized in that: The operation and maintenance data collection module sends event notifications to the operation and maintenance data collection module through an event triggering mechanism, and the process of collecting event data includes: When specific events occur in the target device, server and application, event notifications are sent to the data acquisition module based on preset rules. The specific events include system anomalies, performance bottlenecks and user operations. After receiving the notification, the operation and maintenance data acquisition module immediately starts the collection process, parses and processes the data in the event notification, and extracts key event information. The information is then stored in the database for subsequent analysis, monitoring and alarm operations.

5. The intelligent operation and maintenance management system according to claim 4, characterized in that: The operation and maintenance data processing module formulates a unified technical monitoring standard and adopts an anomaly detection algorithm based on an autoencoder to achieve comprehensive monitoring of the technical environment and real-time data analysis. The process includes: The operation and maintenance data processing module formulates a unified technical monitoring standard to make the data collected from network devices, Windows servers and applications consistent and comparable. The unified technical monitoring standard includes the format, content, collection frequency and storage method of the data, defines the data type, data format, collection time interval and data transmission protocol provided by various data sources, so that all monitored objects output data according to unified specifications, requiring the data to be complete and without missing, the timestamp to be accurate to the second, and the data delay to be no more than 5 minutes. At the same time, a data verification mechanism is established to mark and process abnormal and unreasonable data, formulate data cleaning rules, deduplicate, denoise and format convert the collected raw data, establish a data integration process, clean, integrate and normalize the data from different data sources to form a unified data view, and build an anomaly detection model based on the autoencoder. The input layer of the autoencoder receives the data collected from the operation and maintenance data. The raw data transmitted by the collection module includes performance data, log data and event data of network devices, Windows servers and applications. The input layer passes the data to the next layer for processing. The neurons in the hidden layer of the autoencoder are responsible for extracting the feature representation of the input data, mapping the input data to a low-dimensional coding space through nonlinear transformation, forming a coded representation of the data, and enabling the autoencoder to learn the normal feature mode of the data. The output layer of the autoencoder receives the coded representation transmitted by the hidden layer, and reconstructs it into an approximate value of the input data through decoding. The reconstruction error is calculated by comparing the difference between the reconstructed data and the original input data. If the reconstruction error exceeds the preset error threshold, it indicates that there is an abnormality in the data. If the reconstruction error is within the preset error threshold, it indicates that the data is normal. At the same time, the output layer passes the reconstructed data and the reconstruction error to the data analysis module for the next step of judgment and processing.

6. The intelligent operation and maintenance management system according to claim 5, characterized in that: The operation and maintenance data processing module sets intelligent alarm and warning rules according to the analysis results of the anomaly detection model of the autoencoder. When anomalies and failures are detected, the process of issuing warning information in a timely manner includes: When new data is input into the trained autoencoder, if the input data is normal, the output layer reconstructs a result similar to the input data, which is judged as normal data and does not trigger an early warning. If the input data contains anomalies, the reconstruction error increases significantly and is judged as abnormal data, thereby triggering the anomaly detection mechanism. The operation and maintenance data analysis module immediately generates an early warning message, which includes the type, level, occurrence time and information of related equipment and applications. The early warning message is sent to relevant operation and maintenance personnel by email.

7. An intelligent operation and maintenance management system according to claim 6, characterized in that: The process of displaying the analysis results in a graphical manner by the operation and maintenance data analysis module includes: For the output results of the autoencoder anomaly detection model, a line chart is used to show the changing trend of the data. When the reconstruction error increases significantly, it is highlighted with different marks. At the same time, an interactive function is provided to allow operation and maintenance personnel to obtain detailed information by clicking on the data points in the chart.

Images