A Comprehensive Operation and Maintenance Management System and Method Based on AIOps

Abstract

The present invention relates to the technical field of operation and maintenance management, and specifically relates to an integrated operation and maintenance management system and method based on AIOps; the method includes: collecting multi-source IT operation and maintenance data and completing standardized preprocessing to form operation and maintenance data input in a unified format; classifying and storing the preprocessed data using a hybrid storage architecture, and implementing data synchronization through a cross-database synchronization tool to break data silos; performing intelligent analysis on the data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data; generating operation and maintenance decisions according to the analysis results, and performing automatic repair, work order allocation or resource scheduling operations; outputting operation and maintenance management results, presenting data and decision execution situations through a visualization interface, and providing report generation and management configuration functions; through a full-process automated data processing and intelligent operation and maintenance decision-making mechanism with multi-module collaboration, the technical problems of slow fault response, weak data integration, low accuracy of risk prediction and low operation and maintenance efficiency in the traditional operation and maintenance mode are solved.

Description

Technical Field

[0001] This invention relates to the field of operation and maintenance management technology, and in particular to a comprehensive operation and maintenance management system and method based on AIOps. Background Technology

[0002] With the rapid development of information technology, enterprise IT architectures are evolving rapidly towards distributed and cloud-native architectures. The number of IT resources, such as servers, network devices, databases, and microservice applications, is growing exponentially. Traditional operation and maintenance (O&M) management methods are no longer adequate to meet the complex needs of current IT systems. On the one hand, manual O&M is extremely inefficient. O&M personnel need to monitor dozens of metrics, such as CPU utilization, memory usage, and network latency, on hundreds or thousands of devices simultaneously. Manual troubleshooting requires comparing logs with metric data one by one, resulting in a mean time to report (MTTR) often exceeding 4 hours. Furthermore, human error can easily lead to inaccurate fault location, potentially causing business system outages. On the other hand, traditional O&M... The data is fragmented, with Zabbix used for server monitoring, ELK for log analysis, and JMeter for performance testing. Each tool stores data in an independent database, creating data silos. For example, network device alarm data cannot be linked with application error logs for analysis, making it difficult for operations and maintenance personnel to grasp the overall picture of IT system operation. In addition, traditional systems lack the ability to deeply mine historical data, storing only the last three months of operational data, and cannot predict failure risks based on historical data. For example, they cannot identify potential downtime risks caused by server memory leaks in advance, and can only deal with them passively after a failure occurs, which cannot meet the "99.99% availability" requirements of IT systems in industries such as finance and e-commerce.

[0003] Among the aforementioned existing technologies, especially for enterprises with large IT resources and high business continuity requirements, the shortcomings of traditional operation and maintenance models in terms of fault response speed, data integration capabilities, and risk prediction accuracy are particularly prominent. Therefore, how to achieve automated data collection, intelligent fault analysis, and accurate risk prediction in IT operation and maintenance, while breaking down data silos to achieve full data linkage, has become the core technical problem to solve the current low operation and maintenance efficiency and poor system stability. Summary of the Invention

[0004] The purpose of this invention is to provide an integrated operation and maintenance management system and method based on AIOps, which aims to solve the technical problems of how to achieve automated data collection, intelligent fault analysis, and accurate risk prediction in IT operation and maintenance, while breaking down data silos to achieve full data linkage, thus solving the core technical problems of low operation and maintenance efficiency and poor system stability.

[0005] To achieve the above objectives, this invention employs an AIOps-based integrated operation and maintenance management method, comprising the following steps:

[0006] Collect IT operations and maintenance data from multiple sources and complete standardized preprocessing to form a unified format for operations and maintenance data input;

[0007] A hybrid storage architecture is used to classify and store preprocessed data, and data synchronization is achieved through cross-database synchronization tools to break down data silos.

[0008] Based on the TensorFlow framework, intelligent analysis is performed on the data to generate fault diagnosis results and performance trend prediction data.

[0009] Based on the analysis results, generate operation and maintenance decisions and execute automatic repair, work order allocation or resource scheduling operations.

[0010] It outputs operation and maintenance management results, presents data and decision execution status through a visual interface, and provides report generation and management configuration functions.

[0011] Among the steps involved in collecting multi-source IT operations and maintenance data and completing standardized preprocessing to form a unified format for operations and maintenance data input:

[0012] Deploy a lightweight agent to collect server metrics in real time. Use the API call module to periodically obtain network device port traffic, database connection count, and application request success rate data. Use a Flink-based log collection tool to collect device and application logs. Server metrics include CPU utilization, memory usage, and disk I / O. The API call module is compatible with SNMP, JDBC, and HTTP protocols.

[0013] All collected data is converted to JSON format and null and duplicate values ​​are removed. The collection frequency can be configured from 1 second to 5 minutes. Abnormal logs are retained through log filtering rules to reduce redundancy.

[0014] Among the steps involved, the preprocessed data was categorized and stored using a hybrid storage architecture, and data synchronization was achieved through cross-database synchronization tools to break down data silos:

[0015] Time-series metrics, structured data, and unstructured data are stored using InfluxDB, MySQL, and HDFS, respectively.

[0016] The DataX tool is used to configure scheduled synchronization tasks to achieve cross-database linkage. The default synchronization frequency is 5 minutes, and the cycle can be adjusted according to the importance of the data.

[0017] In the step of performing intelligent analysis on data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data:

[0018] The data preprocessing module completes the mean imputation of missing values, Min-Max normalization of indicators, and extraction of time series features. An anomaly detection is performed by a fusion model of SVM supervised learning and DBSCAN unsupervised learning. An alarm is triggered when the anomaly probability exceeds 85%.

[0019] By using Bayesian network causal analysis and association rule mining to locate the root cause of the fault, and using an LSTM time series prediction model to predict the performance trend 24 hours in advance.

[0020] Among these steps, the process of generating operational decisions based on the analysis results and executing automatic repair, work order allocation, or resource scheduling operations includes:

[0021] The fault handling rule engine is invoked to issue repair instructions for faults that can be automatically repaired, and to generate maintenance work orders containing fault causes, scope of impact, and handling suggestions for faults that require manual intervention.

[0022] The minimum load migration algorithm is used to migrate high-load server services to idle nodes.

[0023] Among the steps, the output of operation and maintenance management results, the presentation of data and decision execution status through a visual interface, and the provision of report generation and management configuration functions are as follows:

[0024] The real-time operational status of IT resources is displayed through a visual dashboard;

[0025] Generate periodic reports (daily / weekly / monthly) based on the operational status of IT resources;

[0026] It provides an interface for multi-dimensional equipment screening, full-process work order processing, and resource scheduling.

[0027] In the process of completing the missing value mean imputation, indicator Min-Max normalization, and time series feature extraction through the data preprocessing module, and performing anomaly detection using a fusion model of SVM supervised learning and DBSCAN unsupervised learning, an alarm is triggered when the anomaly probability exceeds 85%:

[0028] If the same indicator fails to generate an alarm after three consecutive abnormal detections, the detection priority of that indicator will be automatically increased and the collection interval will be shortened to 1 / 3 of the original frequency.

[0029] In the process of completing the missing value mean imputation, indicator Min-Max normalization, and time series feature extraction through the data preprocessing module, and performing anomaly detection using a fusion model of SVM supervised learning and DBSCAN unsupervised learning, an alarm is triggered when the anomaly probability exceeds 85%:

[0030] If the same device triggers the same alarm three times in a row within 24 hours, the system will automatically increase the monitoring priority of the device and reduce the collection frequency to half of the original.

[0031] Among these steps, generating operational decisions based on the analysis results and executing automatic repair, work order allocation, or resource scheduling operations are as follows:

[0032] Work orders are assigned based on the workload of maintenance personnel and notifications are sent via SMS / email / WeChat Work. If a work order is not processed within the specified time, the notification method is escalated every 30 minutes.

[0033] This invention also provides an AIOps-based integrated operation and maintenance management system, including a data acquisition and preprocessing module, a data hybrid storage module, a data intelligent analysis and execution module, an intelligent decision-making module, and an operation and maintenance result display module; wherein:

[0034] The data acquisition and preprocessing module is used to collect multi-source IT operation and maintenance data and complete standardized preprocessing to form a unified format of operation and maintenance data input.

[0035] The data hybrid storage module is used to classify and store preprocessed data using a hybrid storage architecture, and to achieve data synchronization through cross-database synchronization tools, thereby breaking down data silos.

[0036] The data intelligent analysis and execution module is used to perform intelligent analysis on data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data.

[0037] The intelligent decision-making module is used to generate operation and maintenance decisions based on the analysis results and to execute automatic repair, work order allocation or resource scheduling operations.

[0038] The operation and maintenance results display module is used to output operation and maintenance management results, present data and decision execution status through a visual interface, and provide report generation and management configuration functions.

[0039] This invention discloses an AIOps-based integrated operations and maintenance management system and method. It collects multi-source IT operations and maintenance data and performs standardized preprocessing to form a unified format for operations and maintenance data input. A hybrid storage architecture is used to classify and store the preprocessed data, and cross-database synchronization tools are used to achieve data synchronization, breaking down data silos. Based on the TensorFlow framework, intelligent analysis is performed on the data to generate fault diagnosis results and performance trend prediction data. Based on the analysis results, operations and maintenance decisions are generated, and automatic repair, work order allocation, or resource scheduling operations are executed. Operations and maintenance management results are output, presenting data and decision execution status through a visual interface, and providing report generation and management configuration functions.

[0040] As a result, this application significantly improves operation and maintenance efficiency. The automated data collection layer reduces data collection time by 80%, the data analysis layer shortens the average troubleshooting time from 4 hours to 15 minutes, and the intelligent decision-making layer can automatically handle 60% of common faults, significantly reducing the workload of operation and maintenance personnel.

[0041] Completely break down data silos; through a hybrid storage architecture and cross-database synchronization tools, achieve "one-stop" management of server, network, database, and application data, improving data integration efficiency by 90%.

[0042] With accurate fault prediction capabilities, the LSTM time series prediction model identifies potential risks 24 hours in advance. Combined with resource scheduling optimization, it reduces the failure rate of IT systems by 70%, meeting high availability requirements.

[0043] Lowering the technical implementation threshold, the system has a built-in pre-trained AIOps model and standardized fault handling rules, so enterprises do not need to develop it themselves, reducing the technical threshold by 60%; fifth, it is highly adaptable, with each layer of modules designed independently, supporting customized needs such as adding data collection from IoT devices and expanding Transformer prediction models, and can adapt to the evolution of IT architecture from traditional to cloud-native.

[0044] By using a multi-module collaborative, fully automated data processing and intelligent operation and maintenance decision-making mechanism, the technical problems of slow fault response, weak data integration, low accuracy of risk prediction, and low operation and maintenance efficiency in the traditional operation and maintenance model can be solved. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the structural principle of the integrated operation and maintenance management system based on AIOps of the present invention.

[0047] Figure 2 This is a detailed flowchart of the integrated operation and maintenance management method based on AIOps of the present invention.

[0048] Figure 3 This is a flowchart of the steps of the integrated operation and maintenance management method based on AIOps of the present invention.

[0049] Figure 4 This is a flowchart of steps S100 of the present invention.

[0050] Figure 5 This is a flowchart of steps S200 of the present invention.

[0051] Figure 6 This is a flowchart of steps S300 of the present invention.

[0052] Figure 7This is a flowchart of steps S400 of the present invention.

[0053] Figure 8 This is a flowchart of steps S500 of the present invention.

[0054] Figure 9 This is a schematic diagram of the electronic device of the present invention.

[0055] 601 - Data Acquisition and Preprocessing Module, 602 - Data Hybrid Storage Module, 603 - Intelligent Data Analysis and Execution Module, 604 - Intelligent Decision-Making Module, 605 - Operation and Maintenance Result Display Module. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0057] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0058] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0059] Please see Figures 2-8 This invention provides a comprehensive operation and maintenance management method based on AIOps, comprising the following steps:

[0060] S100: Collects multi-source IT operations and maintenance data and completes standardized preprocessing to form a unified format for operations and maintenance data input;

[0061] In this implementation, multi-source IT operations and maintenance data is collected and standardized preprocessed to form a unified format for operations and maintenance data input. The specific process is as follows:

[0062] S101: Deploy a lightweight agent to collect server metrics in real time. It periodically obtains network device port traffic, database connection count, and application request success rate data through the API call module. It uses a Flink-based log collection tool to collect device and application logs. The server metrics include CPU utilization, memory usage, and disk I / O. The API call module is compatible with SNMP, JDBC, and HTTP protocols.

[0063] S102: Convert all collected data into JSON format and remove null and duplicate values. The collection frequency can be configured from 1 second to 5 minutes. Abnormal logs are retained through log filtering rules to reduce redundancy.

[0064] S200: Employs a hybrid storage architecture to classify and store pre-processed data, and uses cross-database synchronization tools to achieve data synchronization, breaking down data silos;

[0065] In this implementation, a hybrid storage architecture is used to categorize and store preprocessed data, and data synchronization is achieved through cross-database synchronization tools, breaking down data silos. The specific process is as follows:

[0066] S201: Time series metrics, structured data, and unstructured data are stored in InfluxDB, MySQL, and HDFS respectively;

[0067] S202: Use the DataX tool to configure scheduled synchronization tasks to achieve cross-database linkage. The default synchronization frequency is 5 minutes, and the cycle can be adjusted according to the importance of the data.

[0068] S300: Based on the TensorFlow framework, it performs intelligent analysis on data to generate fault diagnosis results and performance trend prediction data;

[0069] In this implementation, intelligent analysis is performed on the data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data. The specific process is as follows:

[0070] S301: The data preprocessing module completes the mean imputation of missing values, Min-Max normalization of indicators, and extraction of time series features. An anomaly detection is performed using a fusion model of SVM supervised learning and DBSCAN unsupervised learning. An alarm is triggered when the anomaly probability exceeds 85%.

[0071] S302: Locate the root cause of the fault through Bayesian network causal analysis and association rule mining, and use LSTM time series prediction model to predict the performance trend 24 hours in advance;

[0072] S303: If the same indicator is detected as abnormal three times in a row without triggering an alarm, the detection priority of that indicator will be automatically increased and the collection interval will be shortened to 1 / 3 of the original frequency.

[0073] S304: If the same device triggers the same alarm 3 times in a row within 24 hours, the system will automatically upgrade the monitoring priority of the device and reduce the collection frequency to half of the original.

[0074] Through the above process, during abnormal alarm reception and intelligent diagnosis: when a device triggers an abnormality (such as memory usage exceeding 90%), the system pushes alarm notifications according to the user's preset methods: ① Pop-up reminder (pops up in the lower right corner of the current page, automatically shrinks to the notification bar after 5 seconds); ② SMS / WeChat notification (including abnormal device IP, abnormal indicators, and alarm level); clicking the alarm notification or the "Intelligent Diagnosis" button on the device details page, the system automatically calls the fault location results from the data analysis layer, and displays in a pop-up window: ① Root cause of the fault (such as "process PID12345 memory leak"); ② Scope of impact (such as "only affects the e-commerce coupon distribution module"); ③ Handling suggestions (divided into "automatic repair" and "manual handling").

[0075] S400: Generates operation and maintenance decisions based on analysis results and executes automatic repair, work order allocation, or resource scheduling operations;

[0076] In this implementation, operational decisions are generated based on the analysis results, and automatic repair, work order allocation, or resource scheduling operations are executed. The specific process is as follows:

[0077] S401: Call the fault handling rule engine to issue repair instructions for faults that can be automatically repaired, and generate maintenance work orders containing fault causes, scope of impact, and handling suggestions for faults that require manual intervention;

[0078] S402: Migrate high-load server services to idle nodes based on the minimum load migration algorithm;

[0079] S403: Work orders are assigned based on the workload of maintenance personnel and notifications are sent via SMS / email / WeChat Work. If a work order is not processed within the specified time, the notification method is escalated every 30 minutes.

[0080] Through the above process, the following steps are required for decision-making in this application:

[0081] Automatic repair triggering and result viewing:

[0082] If the intelligent diagnostic results support "automatic repair" (such as high process usage or abnormal port), click the "One-Click Repair" button, and the system will send a repair instruction to the intelligent decision layer. During the repair process, the page will display the "Repairing (progress bar)" status. After the repair is completed, a pop-up window will indicate the result (success / failure): ① Repair successful: Synchronously update the device indicators to the normal range; ② Repair failed: Display the reason for the failure (such as "insufficient permissions") and provide manual handling guidance.

[0083] Manual work order initiation and information filling:

[0084] If manual intervention is required due to an anomaly, click the "Generate Work Order" button. The system will automatically fill in the basic information (abnormal device IP, fault type, and diagnostic results). The user needs to add: ① Work order priority (inherited from alarm priority by default, but can be modified); ② Processing deadline (within 2 hours by default, but can be customized); ③ Additional notes (such as "This server needs to be processed during off-peak hours"). After submitting the work order, the system will synchronize it to the intelligent decision-making layer allocation module in real time and automatically match the maintenance personnel with the lowest load.

[0085] Work order receipt and processing progress update:

[0086] After logging in, regular maintenance personnel will receive a system pop-up notification and see a new work order in the "Work Order Center - Pending" list if one is assigned. Clicking on a work order will take them to the processing page: ① View work order details (fault description, processing suggestions, associated files); ② Click "Start Processing" to update the work order status to "Processing"; ③ During processing, operation logs can be added (e.g., "Execute the free -m command to check memory usage"); ④ After processing is complete, upload a screenshot of the processing (e.g., "Screenshot showing memory usage dropping to 60%) and submit it for review.

[0087] Work order approval and archiving (administrator function):

[0088] The administrator enters the "Work Order Center - Pending Review" list to view the work orders submitted by the operations and maintenance personnel: ① Review the processing results (compare the changes in indicators before and after the fault); ② If the review is approved, update the work order status to "Completed" and archive it to the MySQL work order table; ③ If the review is not approved, fill in the reason for rejection (such as "Memory leak problem not completely resolved"), return the work order to the original handler, and send a rejection notification.

[0089] Resource scheduling viewing and confirmation (administrator function):

[0090] Administrators can click "Resource Scheduling" on the left to enter the scheduling page: ① View the server load heatmap (red indicates high load ≥80%, green indicates low load ≤40%); ② The system recommends a scheduling plan (e.g., "Migrate the high-load server 192.168.1.100 in the Chongqing data center to the idle server 192.168.2.200 in the Chengdu data center"); ③ Click "View Details" to view the impact of the migration (e.g., "E-commerce order query latency may increase by 100ms during the migration"); After confirming that everything is correct, click "Execute Scheduling". The system will automatically complete the business migration and send a result notification after the migration is complete.

[0091] S500: Outputs operation and maintenance management results, presents data and decision execution status through a visual interface, and provides report generation and management configuration functions.

[0092] In this implementation, the operation and maintenance management results are output, and the data and decision execution status are presented through a visual interface, providing report generation and management configuration functions. The specific process is as follows:

[0093] S501: Displays the real-time operational status of IT resources through a visual dashboard;

[0094] S502: Generate periodic reports (daily / weekly / monthly) based on the operational status of IT resources;

[0095] S503: Provides an interface for multi-dimensional equipment screening, full-process work order processing, and resource scheduling.

[0096] Through the above process, this application requires the following operations to be performed when a user logs in:

[0097] Multi-role login and authentication:

[0098] Users access the system address through a browser, and after entering their account and password, administrators trigger two-factor authentication, while ordinary maintenance personnel use single-factor authentication by default. If the wrong password is entered three times in a row, the account will be temporarily locked for 15 minutes. Login will be automatically logged out after a timeout (default 30 minutes of inactivity). The system will synchronously record login logs (including login IP, login time, and role type) to a MySQL structured database for easy auditing and traceability later.

[0099] Initial login configuration:

[0100] When a new user logs in for the first time, a pop-up window guides them through the initial configuration: ① Select the business line you are responsible for (e.g., e-commerce business / financial payment business); ② Set commonly used monitoring metrics (e.g., CPU utilization, memory usage, network latency); ③ Customize alarm notification methods (SMS / email / WeChat Work, multiple selections are allowed); After configuration, the system automatically generates a personalized homepage dashboard, which can be modified later in "Settings - Personalized Configuration".

[0101] Homepage dashboard viewing and indicator acquisition:

[0102] After logging in, you will be directed to the homepage dashboard by default. The page is displayed in sections: "Resource Overview - Anomaly Alarms - Work Order Progress - Performance Trends". ① The Resource Overview section displays the online rate of all IT devices and the availability of core business systems. ② The Anomaly Alarms section displays unprocessed alarms by priority (P1 red / P2 yellow / P3 blue). ③ Clicking on any metric (such as "e-commerce server CPU utilization") will display the trend curve of that metric over the past 1 hour / 24 hours / 7 days (generated based on InfluxDB time series data). You can drag and drop the time axis to filter data for a specific period.

[0103] Filtering and viewing device list details:

[0104] Click "Device Management" in the left navigation bar to enter the device list page, which supports multi-dimensional filtering: ① By device type (server / network switch / database); ② By data center location (Chongqing data center / Chengdu data center); ③ By operating status (online / offline / abnormal); After selecting the target device, enter the device details page, which displays: ① Basic information (device IP, model, business line); ② Real-time operating indicators (refreshed every 10 seconds); ③ Historical fault records (last 3 months); ④ Associated application systems (such as the e-commerce order system deployed on this server).

[0105] The specific details included during work order generation are as follows:

[0106] Operation and maintenance report generation and export:

[0107] Enter the "Report Management" module, which supports the generation of two types of reports: ① Periodic reports (daily / weekly / monthly reports, which by default include resource utilization, fault handling rate, and work order completion rate); ② Custom reports (users select the indicators to be displayed and choose the time range); Click "Generate Report", and the system will generate a report based on MySQL structured data and HDFS unstructured data, supporting: ① Online preview (HTML format); ② Export and download (PDF / Excel format); ③ Sharing to a specified email address (supports multiple email addresses, with report attachments).

[0108] User permission settings and configuration modifications (administrator functions):

[0109] Administrators can access "System Settings - Permission Management" to: ① Add a user (fill in the account, name, department, and assign the role: Administrator / Regular Operations Personnel); ② Modify permissions (e.g., restrict a certain operations personnel to only view equipment in the Chongqing data center); ③ Delete a user (requires secondary confirmation; the user's historical operation logs will be retained after deletion); In "System Settings - Monitoring Configuration", the following can be modified: ① Metric alarm thresholds (e.g., adjust the CPU anomaly threshold from 85% to 90%); ② Data collection frequency (e.g., adjust the core equipment frequency from 10 seconds / time to 5 seconds / time).

[0110] Operation log query and problem feedback:

[0111] All users can access "My - Operation Log" to view their operation records for the past 6 months (including operation time, operation module, operation content, and result). If you encounter any problems during use (such as function abnormalities or operation lag), click the "Feedback" button in the lower right corner of the page and fill in: ① Problem type (function failure / operation suggestion / other); ② Problem description; ③ Upload screenshot (optional). After submission, the system will synchronize the feedback information to the R&D team's ticket system, and notify the user via in-system message after processing.

[0112] As a result, this application significantly improves operation and maintenance efficiency. The automated data collection layer reduces data collection time by 80%, the data analysis layer shortens the average troubleshooting time from 4 hours to 15 minutes, and the intelligent decision-making layer can automatically handle 60% of common faults, significantly reducing the workload of operation and maintenance personnel.

[0113] Completely break down data silos; through a hybrid storage architecture and cross-database synchronization tools, achieve "one-stop" management of server, network, database, and application data, improving data integration efficiency by 90%.

[0114] With accurate fault prediction capabilities, the LSTM time series prediction model identifies potential risks 24 hours in advance. Combined with resource scheduling optimization, it reduces the failure rate of IT systems by 70%, meeting high availability requirements.

[0115] Lowering the technical threshold for implementation, the system has built-in pre-trained AIOps models and standardized fault handling rules, eliminating the need for enterprises to develop their own systems, thus reducing the technical threshold by 60%. Fifth, it is highly adaptable, with each module designed independently to support customized needs such as adding data collection from IoT devices and expanding Transformer prediction models, adapting to the evolution of IT architecture from traditional to cloud-native.

[0116] Corresponding to the aforementioned embodiments of the integrated operation and maintenance management method based on AIOps, this application also provides embodiments of an integrated operation and maintenance management system based on AIOps.

[0117] Figure 1 This is a block diagram of an AIOps-based integrated operation and maintenance management system, illustrated according to an exemplary embodiment. (Refer to...) Figure 1 The system may include: a data acquisition and preprocessing module 601, a data hybrid storage module 602, a data intelligent analysis and execution module 603, an intelligent decision-making module 604, and an operation and maintenance result display module 605, wherein:

[0118] The data acquisition and preprocessing module 601 is used to acquire multi-source IT operation and maintenance data and complete standardized preprocessing to form a unified format of operation and maintenance data input.

[0119] The data hybrid storage module 602 is used to classify and store preprocessed data using a hybrid storage architecture, and to achieve data synchronization through cross-database synchronization tools, thereby breaking down data silos.

[0120] The data intelligent analysis execution module 603 is used to perform intelligent analysis on data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data.

[0121] The intelligent decision-making module 604 is used to generate operation and maintenance decisions based on the analysis results and to execute automatic repair, work order allocation or resource scheduling operations.

[0122] The operation and maintenance result display module 605 is used to output operation and maintenance management results, present data and decision execution status through a visual interface, and provide report generation and management configuration functions.

[0123] In this embodiment, the data acquisition and preprocessing module 601 collects multi-source IT operation and maintenance data and completes standardized preprocessing to form a unified format of operation and maintenance data input; the data hybrid storage module 602 adopts a hybrid storage architecture to classify and store the preprocessed data, and realizes data synchronization through cross-database synchronization tools to break down data silos; the data intelligent analysis and execution module 603 performs intelligent analysis on the data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data; the intelligent decision-making module 604 generates operation and maintenance decisions based on the analysis results and executes automatic repair, work order allocation or resource scheduling operations; the operation and maintenance result display module 605 outputs operation and maintenance management results, presents data and decision execution status through a visual interface, and provides report generation and management configuration functions.

[0124] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0125] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0126] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the AIOps-based integrated operation and maintenance management method described above. Figure 9 The diagram shown is a hardware structure diagram of any device with data processing capabilities within an AIOps-based integrated operation and maintenance management system provided in an embodiment of the present invention, except... Figure 9 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0127] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the AIOps-based integrated operation and maintenance management method described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0128] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0129] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A comprehensive operation and maintenance management method based on AIOps, characterized in that, Includes the following steps: Collect IT operations and maintenance data from multiple sources and complete standardized preprocessing to form a unified format for operations and maintenance data input; A hybrid storage architecture is used to classify and store preprocessed data, and data synchronization is achieved through cross-database synchronization tools to break down data silos. Based on the TensorFlow framework, intelligent analysis is performed on the data to generate fault diagnosis results and performance trend prediction data. Based on the analysis results, generate operation and maintenance decisions and execute automatic repair, work order allocation or resource scheduling operations. Output operation and maintenance management results, present data and decision execution status through a visual interface, and provide report generation and management configuration functions.

2. The integrated operation and maintenance management method based on AIOps as described in claim 1, characterized in that, In the steps of collecting multi-source IT operations and maintenance data and completing standardized preprocessing to form a unified format for operations and maintenance data input: Deploy a lightweight agent to collect server metrics in real time. Use the API call module to periodically obtain network device port traffic, database connection count, and application request success rate data. Use a Flink-based log collection tool to collect device and application logs. Server metrics include CPU utilization, memory usage, and disk I / O. The API call module is compatible with SNMP, JDBC, and HTTP protocols. All collected data is converted to JSON format and null and duplicate values ​​are removed. The collection frequency can be configured from 1 second to 5 minutes. Abnormal logs are retained through log filtering rules to reduce redundancy.

3. The integrated operation and maintenance management method based on AIOps as described in claim 1, characterized in that, In the process of adopting a hybrid storage architecture to classify and store preprocessed data, and using cross-database synchronization tools to achieve data synchronization and break down data silos: Time-series metrics, structured data, and unstructured data are stored using InfluxDB, MySQL, and HDFS, respectively. The DataX tool is used to configure scheduled synchronization tasks to achieve cross-database linkage. The default synchronization frequency is 5 minutes, and the cycle can be adjusted according to the importance of the data.

4. The integrated operation and maintenance management method based on AIOps as described in claim 1, characterized in that, In the steps of performing intelligent analysis on data based on the TensorFlow framework to generate fault diagnosis results and performance trend prediction data: The data preprocessing module completes the mean imputation of missing values, Min-Max normalization of indicators, and extraction of time series features. An anomaly detection is performed by a fusion model of SVM supervised learning and DBSCAN unsupervised learning. An alarm is triggered when the anomaly probability exceeds 85%. By using Bayesian network causal analysis and association rule mining to locate the root cause of the fault, and using an LSTM time series prediction model to predict the performance trend 24 hours in advance.

5. The integrated operation and maintenance management method based on AIOps as described in claim 1, characterized in that, In the steps of generating operational decisions based on analysis results and executing automatic repair, work order allocation, or resource scheduling operations: The fault handling rule engine is invoked to issue repair instructions for faults that can be automatically repaired, and to generate maintenance work orders containing fault causes, scope of impact, and handling suggestions for faults that require manual intervention. The minimum load migration algorithm is used to migrate high-load server services to idle nodes.

6. The integrated operation and maintenance management method based on AIOps as described in claim 1, characterized in that, In the steps of outputting operation and maintenance management results, presenting data and decision execution status through a visual interface, and providing report generation and management configuration functions: The real-time operational status of IT resources is displayed through a visual dashboard; Generate periodic reports (daily / weekly / monthly) based on the operational status of IT resources; It provides an interface for multi-dimensional equipment screening, full-process work order processing, and resource scheduling.

7. The integrated operation and maintenance management method based on AIOps as described in claim 4, characterized in that, In the steps of completing the missing value mean imputation, index Min-Max normalization, and time series feature extraction through the data preprocessing module, and performing anomaly detection using a fusion model of SVM supervised learning and DBSCAN unsupervised learning, an alarm is triggered when the anomaly probability exceeds 85%: If the same indicator fails to generate an alarm after three consecutive abnormal detections, the detection priority of that indicator will be automatically increased and the collection interval will be shortened to 1 / 3 of the original frequency.

8. The integrated operation and maintenance management method based on AIOps as described in claim 4, characterized in that, In the steps of completing the missing value mean imputation, index Min-Max normalization, and time series feature extraction through the data preprocessing module, and performing anomaly detection using a fusion model of SVM supervised learning and DBSCAN unsupervised learning, an alarm is triggered when the anomaly probability exceeds 85%: If the same device triggers the same alarm three times in a row within 24 hours, the system will automatically increase the monitoring priority of the device and reduce the collection frequency to half of the original.

9. The integrated operation and maintenance management method based on AIOps as described in claim 5, characterized in that, In the steps of generating operational decisions based on analysis results and executing automatic repair, work order allocation, or resource scheduling operations: Work orders are assigned based on the workload of maintenance personnel and notifications are sent via SMS / email / WeChat Work. If a work order is not processed within the specified time, the notification method is escalated every 30 minutes.

10. An AIOps-based integrated operation and maintenance management system, employing the AIOps-based integrated operation and maintenance management method as described in claim 1, characterized in that, The system includes a data acquisition and preprocessing module, a data hybrid storage module, a data intelligent analysis and execution module, an intelligent decision-making module, and an operation and maintenance result display module. Specifically: The data acquisition and preprocessing module collects multi-source IT operation and maintenance data and performs standardized preprocessing to form a unified format for operation and maintenance data input; The data hybrid storage module uses a hybrid storage architecture to classify and store preprocessed data, and achieves data synchronization through cross-database synchronization tools to break down data silos; The data intelligent analysis and execution module performs intelligent analysis on the data based on the TensorFlow framework, generating fault diagnosis results and performance trend prediction data; The intelligent decision-making module generates operation and maintenance decisions based on the analysis results, and executes automatic repair, work order allocation, or resource scheduling operations; The operation and maintenance result display module outputs operation and maintenance management results, presents data and decision execution status through a visual interface, and provides report generation and management configuration functions.

Images