An operation and maintenance management system based on cloud computing services

Abstract

The present invention discloses an operation and maintenance management system based on cloud computing services, which belongs to the field of operation and maintenance management technology. It includes a resource management module connected by wireless communication, a monitoring and alarm module, a fault diagnosis module, a performance optimization module and a security management module; the resource management module is configured to uniformly manage cloud computing resources, and the monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue alarms in time when abnormalities occur. The present invention can effectively integrate data scattered in different cloud services and applications by establishing a unified resource management module. The resource management module realizes the full life cycle management of resources through the unified management of computing, storage and network resources, improves resource utilization and reduces operating costs. The performance indicators of cloud computing resources are monitored in real time through the monitoring and alarm module, and customized monitoring thresholds and alarm rules are supported to ensure the stability and reliability of the system.

Description

Technical Field

[0001] The present invention relates to the field of operation and maintenance management technology, and in particular to an operation and maintenance management system based on cloud computing services. Background Art

[0002] The Operations and Maintenance Management System (O&M) based on cloud computing services is a solution that uses cloud computing technology to support and optimize enterprise operations and maintenance management. It helps enterprises achieve efficient management, monitoring and operation of cloud resources through centralized, automated and intelligent methods, and improves the reliability, security and flexibility of cloud services.

[0003] The operation and maintenance management system based on cloud computing services has been widely used and developed in recent years. It has been widely used in enterprise IT operation and maintenance to manage the internal IT infrastructure of the enterprise, including servers, network equipment and applications. Industrial equipment operation and maintenance remotely monitors and maintains industrial equipment to improve production efficiency. Cloud service operation and maintenance manages applications and services in the cloud environment to ensure their stability and security. At present, cloud computing technology is becoming more and more mature, and the functions of infrastructure (IaaS), platform (PaaS) and software (SaaS) services are constantly enhanced to support more complex operation and maintenance management needs; more and more operation and maintenance management systems adopt automation tools and artificial intelligence (AI) technologies, such as machine learning and big data analysis, to improve fault detection, prediction and response capabilities. More and more operation and maintenance management systems adopt automation tools and artificial intelligence (AI) technologies, such as machine learning and big data analysis, to improve fault detection, prediction and response capabilities. With the increasingly stringent data security and privacy regulations, cloud service providers continue to strengthen security measures and provide more comprehensive security solutions to optimize the user experience of operation and maintenance systems.

[0004] However, while the operation and maintenance management system based on cloud computing services is developing rapidly, it is also facing some challenges and problems: 1. Data between different cloud services and applications are often scattered in different systems and platforms, resulting in asymmetric and incomplete information, lack of unified data management and integration tools, making it impossible to share resources and centrally manage them, and even more impossible to obtain comprehensive information related to faults. 2. Although cloud service providers usually provide high-availability services, they may still be affected by network failures, service interruptions and other failures, resulting in business interruptions and system failures that cannot be quickly located and resolved. 3. Improper monitoring configuration, or the monitoring tools used are not comprehensive enough to cover all key services and resources, and cannot capture all key indicators in real time, resulting in a failure to respond in a timely manner after the failure occurs.

[0005] There is an urgent need for an enterprise operation and maintenance management system that can achieve data sharing and unified management, and can accurately diagnose and resolve faults. Summary of the invention

[0006] In order to solve the problems existing in the above background technology, the present invention adopts the following technical solutions:

[0007] An operation and maintenance management system based on cloud computing services includes a resource management module connected by wireless communication, a monitoring and alarm module, a fault diagnosis module, a performance optimization module and a security management module; wherein the resource management module is configured to uniformly manage cloud computing resources, specifically including the following steps:

[0008] S1. Integrate and synchronize cloud computing resources. Use the APIs provided by various cloud services to regularly collect cloud computing resource data from different platforms to achieve automatic synchronization of cloud computing resource data. Establish a data warehouse to centrally store cloud computing resource data from different cloud services.

[0009] S2. Label and classify cloud computing resources, label different cloud computing resources stored in the database with unified labels, formulate unified classification standards, and classify cloud computing resource types by resource type, which are classified into: computing resources, storage resources, and network resources;

[0010] S3: Unified management of tagged and classified cloud computing resource data, including resource creation, configuration, deletion, expansion and reduction operations.

[0011] Preferably, the tagged and classified cloud computing resource data is managed in a unified manner, including resource creation, configuration, deletion, expansion and reduction operations, specifically including:

[0012] The user submits a resource creation request through the management interface or API, specifying the resource type, performance, and capacity parameters; the system allocates resources in the cloud platform according to the user's resource creation request, creates resources, and returns relevant resource creation information;

[0013] The user configures the created resources and configures the corresponding resource permissions and parameter settings for the created resources;

[0014] The user initiates a deletion request through the management interface or API and specifies the resources to be deleted; the system performs the deletion operation based on the received deletion request and releases the related resources and configurations;

[0015] The user initiates a capacity expansion request through the management interface or API and specifies the resources to be expanded. The system configures the expansion of resources based on the expansion request.

[0016] The user initiates a scaling request through the management interface or API and specifies the resources to be scaled down; the system scales down the resources according to the scaling request.

[0017] Preferably, the monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue alarms in time when an abnormality occurs; the performance indicators of cloud computing resources include performance indicators of computing resources, performance indicators of storage resources and performance indicators of network resources.

[0018] Preferably, the monitoring alarm module includes a monitoring indicator unit, an alarm setting unit, a data analysis unit and an alarm notification unit connected in communication; wherein,

[0019] The monitoring indicator unit is used to monitor the performance indicators of cloud computing resources in real time;

[0020] The alarm setting unit is used to set the alarm threshold, alarm rules and alarm channels of the performance indicators;

[0021] The data analysis unit is used to analyze the performance indicator data to determine whether to trigger the alarm condition of the alarm setting unit;

[0022] The alarm notification unit is configured to trigger an alarm notification based on the analysis result of the data analysis unit, when the analysis result shows that the performance indicator reaches the alarm condition, and send the alarm notification through a preset channel.

[0023] Preferably, the monitoring alarm module also includes integrated alarm settings, which integrates alarm thresholds, alarm rules and alarm channels into one alarm setting unit.

[0024] Preferably, the monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue an alarm in time when an abnormality occurs, including the following steps:

[0025] S10, obtaining the type of cloud computing resources to be monitored; the cloud computing resource types include computing resources, storage resources and network resources;

[0026] S20, obtaining real-time performance indicator data of the monitored cloud computing resources; extracting the real-time performance indicator data of the cloud computing resources according to the monitoring indicator unit;

[0027] S30, analyzing the acquired real-time performance indicator data to generate an abnormal performance indicator data set;

[0028] S40. Issue an alarm based on the abnormal performance indicator data set.

[0029] Preferably, the fault diagnosis module is configured to use machine learning and data analysis technology to intelligently diagnose and analyze faults in the operation and maintenance management system based on cloud computing services, locate the root cause of the fault and give corresponding guidance and suggestions; specifically including:

[0030] System data collection, including collection of log data, performance indicator data of cloud computing resources, system event data, and historical fault data;

[0031] System data processing, including data preprocessing and feature extraction;

[0032] Model training, including algorithm selection, model training, and model evaluation;

[0033] Fault detection, including real-time monitoring and anomaly detection;

[0034] Fault root cause location, including fault classification, root cause analysis and correlation analysis;

[0035] Generate fault solutions based on the fault type and root cause analysis results.

[0036] Preferably, data preprocessing includes data cleaning of the collected log data, performance indicator data of cloud computing resources, system event data and historical fault data, and removing outliers, duplicate data and noise from the log data, performance indicator data of cloud computing resources, system event data and historical fault data; data preprocessing also includes integrating and normalizing the cleaned data to generate a unified data format.

[0037] Preferably, the performance optimization module is configured to analyze and evaluate the performance of cloud computing resources, identify performance bottlenecks and optimization points, and provide performance optimization suggestions and solutions, including:

[0038] Configure monitoring tools, set monitoring indicators and thresholds, and collect performance data of cloud computing resources in real time;

[0039] Generate performance reports of cloud computing resources regularly, and evaluate the performance trends of cloud computing resources by combining historical data of cloud computing resources;

[0040] Design test scenarios, simulate different user behaviors, and generate performance test results for cloud computing resources;

[0041] Based on the performance test results, identify the deficiencies in parameter configuration, modify or adjust the parameter configuration of the application server; restart the service and monitor the performance changes of the modified or adjusted cloud computing resources;

[0042] Based on the performance changes of the modified or adjusted cloud computing resources, performance optimization suggestions and solutions are proposed.

[0043] Preferably, the security management module is configured to manage and monitor the security of cloud computing resources, including user authentication, authority management, access control, security auditing and real-time monitoring of security events.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention can effectively integrate data scattered in different cloud services and applications by establishing a unified resource management module. The resource management module realizes the full life cycle management of resources through unified management of computing, storage and network resources. Through effective creation, configuration, deletion, expansion and reduction operations, enterprises can flexibly respond to business needs, improve resource utilization and reduce operating costs.

[0046] The present invention monitors the performance indicators of cloud computing resources in real time through the monitoring and alarm module, supports customized monitoring thresholds and alarm rules, and ensures the stability and reliability of the system. Through an effective alarm notification mechanism and data analysis, it helps the operation and maintenance team to identify and solve potential problems in a timely manner, thereby improving the utilization rate of cloud computing resources and the quality of service management.

[0047] The present invention sets up a systematic process through the fault diagnosis module, combines machine learning and data analysis technology, and realizes intelligent diagnosis and analysis of system faults. This process can not only quickly locate the root cause of the fault, but also provide effective guidance and suggestions, thereby improving the reliability and flexibility of the system.

[0048] The security management module of the present invention ensures the security of cloud resources by comprehensively implementing measures such as user authentication, authority management, access control, and security auditing. Real-time monitoring of security events and timely adoption of protective measures can effectively prevent and respond to potential security threats, further improving the reliability of the operation and maintenance management system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram of the structure of an operation and maintenance management system based on cloud computing services provided by the present invention;

[0051] Figure 2 A schematic diagram of a process of a resource management module provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the structure of a monitoring and alarm module provided in one embodiment of the present invention;

[0053] Figure 4 A process diagram of a monitoring and alarm module provided by an embodiment of the present invention;

[0054] Figure 5 A process diagram of a fault diagnosis module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0056] See also Figure 1 The present invention provides an operation and maintenance management system based on cloud computing services, including a resource management module connected by wireless communication, a monitoring and alarm module, a fault diagnosis module, a performance optimization module and a security management module.

[0057] The resource management module is configured to manage cloud computing resources in a unified manner and realize the full life cycle management of resources. Cloud computing resources include computing resources, storage resources, and network resources. Computing resources include virtual machines (VMs), containers (such as Docker, Kubernetes, etc.), and serverless computing (such as AWS Lambda, Azure Functions, etc.). Storage resources include block storage (such as Amazon EBS, Azure Disk Storage), object storage (such as Amazon S3, Google CloudStorage, etc.), and file storage (such as Amazon EFS, Azure Files, etc.). Network resources include virtual networks, subnets, routers, firewalls, etc.

[0058] The resource management module is configured to manage cloud computing resources in a unified manner, including integrating cloud computing resource data scattered across different cloud services and applications to achieve unified management. Figure 2 As shown, the specific steps include:

[0059] S1. Integrate and synchronize cloud computing resources. Use the APIs provided by each cloud service to regularly collect cloud computing resource data from different platforms to achieve automatic synchronization of cloud computing resource data. Establish a data warehouse to centrally store cloud computing resource data from different cloud services for easy subsequent analysis and management.

[0060] S2. Label and classify cloud computing resources. Label different cloud computing resources stored in the database with unified labels to facilitate classification and search, helping operation and maintenance personnel to quickly locate required resources. At the same time, formulate a unified classification standard to classify cloud computing resource types by resource type, which are classified into: computing resources, storage resources and network resources.

[0061] S3: Unified management of tagged and classified cloud computing resource data, including resource creation, configuration, deletion, expansion, and reduction. Specifically:

[0062] 1. The user submits a resource creation request through the management interface or API, specifying parameters such as resource type, performance, and capacity; the system allocates resources in the cloud platform according to the user's resource creation request, creates resources, and returns relevant resource creation information. For example:

[0063] Computing resource creation: Users submit creation requests through the management interface or API, specifying parameters such as computing resource type, CPU, memory, operating system, etc. The system allocates resources in the cloud platform based on the request, creates a virtual machine or container instance, and returns the instance ID.

[0064] Storage resource creation: The user selects a storage type (such as block storage or object storage), specifies capacity and performance requirements, and submits a creation request. The system creates a storage volume or bucket on the backend and returns relevant information.

[0065] Network resource creation: The user defines the network settings, including network name, CIDR, subnet, etc., and submits a creation request. The system configures the virtual network and creates the corresponding routes and security groups.

[0066] 2. The user configures the created resources, and configures the corresponding resource permissions and parameter settings for the created resources. For example:

[0067] Computing resource configuration: Users can configure the created computing resources through the management interface or API, including installing software, setting up security groups, etc. The system supports automated configuration through configuration management tools (such as Ansible, Chef, etc.).

[0068] Storage resource configuration: Users can set access permissions, backup policies, and performance parameters for storage resources. The system provides monitoring tools to monitor storage performance and usage in real time.

[0069] Network resource configuration: Users can configure network policies, firewall rules, load balancing and other settings. The system supports dynamic adjustment of network configuration to meet business needs.

[0070] 3. The user initiates a deletion request through the management interface or API, and specifies the resources to be deleted; the system performs the deletion operation according to the received deletion request, and releases the related resources and configurations. In an embodiment of the present invention, before the system deletes the resources, it also includes a dependency check to ensure that no other resources depend on the deleted resources to avoid causing failures.

[0071] 4. The user initiates a capacity expansion request through the management interface or API and specifies the resources to be expanded; the system configures the expansion of resources based on the capacity expansion request. For example:

[0072] Computing resource expansion: Users can submit expansion requests through the management interface or API to increase CPU, memory or number of instances. The system automatically adjusts resource configuration based on the application and ensures business continuity during the expansion process.

[0073] Storage resource expansion: Users can increase storage capacity by submitting expansion requests through the management interface or API. The system dynamically expands the capacity of the storage volume in the backend to ensure data integrity.

[0074] Network resource expansion: Users can add subnets, routes or firewall rules and submit expansion requests. The system automatically adjusts the network configuration to ensure the normal operation of the network after expansion.

[0075] 5. The user initiates a scaling request through the management interface or API and specifies the resources to be scaled down; the system scales down the resources according to the scaling request. For example:

[0076] Computing resource scaling: Users can submit scaling requests through the management interface or API to reduce CPU, memory, or the number of instances. Before executing the scaling, the system checks resource usage to ensure that it will not affect running services.

[0077] Storage resource reduction: Users can reduce storage capacity (depending on the specific storage type) by submitting a reduction request. Before reducing capacity, the system ensures that data is not lost and performs necessary backups.

[0078] Network resource reduction: Users can delete subnets, routes, or firewall rules that are no longer needed and submit reduction requests. When the system performs reduction, it ensures that the normal operation of other network resources is not affected.

[0079] The present invention can effectively integrate data scattered in different cloud services and applications by establishing a unified resource management module. The resource management module realizes the full life cycle management of resources through unified management of computing, storage and network resources. Through effective creation, configuration, deletion, expansion and reduction operations, enterprises can flexibly respond to business needs, improve resource utilization and reduce operating costs.

[0080] The monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue alarms in time when an abnormality occurs. The performance indicators of cloud computing resources include performance indicators of computing resources, storage resources and network resources. Specifically, the performance indicators of computing resources include CPU utilization, memory utilization and load balancing, etc.; the performance indicators of storage resources include disk I / O, storage utilization, read and write speed, delay, etc.; the performance indicators of network resources include network traffic, bandwidth utilization, delay and packet loss rate, etc.

[0081] like Figure 3 As shown, the monitoring alarm module includes a monitoring indicator unit, an alarm setting unit, a data analysis unit and an alarm notification unit connected in communication. Among them, the monitoring indicator unit is used to monitor the performance indicators of cloud computing resources in real time; the alarm setting unit is used to set the alarm threshold, alarm rules and alarm channels of the performance indicators; the data analysis unit is used to analyze the performance indicator data to determine whether to trigger the alarm condition of the alarm setting unit; the alarm notification unit is configured to trigger the alarm notification according to the analysis result of the data analysis unit, when the analysis result shows that the performance indicator reaches the alarm condition, and send the alarm notification through the preset channel.

[0082] The alarm setting unit is used to set the alarm threshold, alarm rules and alarm channels of performance indicators, including:

[0083] Set the alarm threshold of the performance indicator. The alarm threshold refers to the situation where an alarm is required when the performance indicator is within a certain threshold range, including static threshold and dynamic threshold. The static threshold refers to the fixed threshold set manually by the user, such as triggering an alarm when the CPU usage exceeds 80%; the dynamic threshold refers to the system automatically adjusting the threshold based on historical data and trend analysis to adapt to the changing workload. In a specific embodiment of the present invention, the static thresholds of the performance indicators of computing resources, storage resources and network resources are respectively set to:

[0084] Computing resources:

[0085] CPU usage: >85%,

[0086] Memory usage: >90%,

[0087] Load balancing: >70%,

[0088] Storage resources:

[0089] Disk I / O: >1000 operations / second,

[0090] Storage utilization: >90%,

[0091] Read and write speed: <50 MB / s,

[0092] Latency: >200 ms,

[0093] Online resources:

[0094] Network traffic: >1000 MB,

[0095] Bandwidth usage: >80%,

[0096] Latency: >100 ms,

[0097] Packet loss rate: >5%.

[0098] Set alarm rules. Alarm rules indicate that an alarm is triggered under certain conditions. In the embodiment of the present invention, the alarm rule is that an alarm is triggered when the alarm threshold is exceeded. It mainly includes: single indicator alarm: when a certain monitoring indicator exceeds the preset threshold, an alarm is triggered. For example, an alarm is sent when the CPU usage exceeds 85%; composite indicator alarm: when multiple indicators meet specific conditions at the same time, an alarm is triggered. For example, an alarm is sent when the CPU usage exceeds 85% and the memory usage exceeds 80%; time window alarm: when the threshold is continuously exceeded within a specific time period, an alarm is triggered. For example, an alarm is sent when the CPU usage exceeds 90% for 5 minutes.

[0099] Setting alarm rules also includes setting different alarm levels, including reminders, warnings, and severe warnings. Specifically, when the performance index exceeds the alarm threshold by 10%, it is set as a reminder; when the performance index exceeds the alarm threshold by 10%-50%, it is set as a warning; when the performance index exceeds the alarm threshold by 50%, it is set as a severe warning.

[0100] Set up an alarm channel. The alarm channel is configured to notify relevant personnel through relevant channels when an alarm is triggered. The channels include email, SMS, Webhook, etc.

[0101] The monitoring alarm module also includes integrated alarm settings, which integrate alarm thresholds, alarm rules and alarm channels into an alarm setting unit for subsequent use.

[0102] The data analysis unit is used to analyze the performance indicator data and determine whether to trigger the alarm condition of the alarm setting unit; the data analysis unit can help users better understand resource usage and performance trends. Specifically, it includes: historical data storage, which stores monitoring data and alarm records to support subsequent analysis and auditing; trend analysis, which performs trend analysis based on historical data to help users predict resource requirements and performance changes; visual reports, which generate visual reports to display resource usage, alarm history and performance trends to facilitate decision-making.

[0103] Specifically, the monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue alarms in time when an abnormality occurs, such as Figure 4 As shown, the following steps are included:

[0104] S10. Obtain the type of cloud computing resources to be monitored; the cloud computing resource types include computing resources, storage resources, and network resources.

[0105] S20, obtaining real-time performance indicator data of the monitored cloud computing resources; extracting the real-time performance indicator data of the cloud computing resources according to the monitoring indicator unit. For computing resources, the obtained performance indicators include CPU usage, memory usage, load balancing, etc.; for storage resources, the obtained performance indicators include disk I / O, storage usage, read and write speed, latency, etc.; for network resources, the obtained performance indicators include network traffic, bandwidth usage, latency, and packet loss rate, etc.

[0106] S30. Analyze the acquired real-time performance indicator data to generate an abnormal performance indicator data set; specifically, compare the acquired real-time performance indicator data with the alarm threshold and alarm rule of the corresponding cloud computing resource set in the alarm setting unit, mark the performance indicator data that does not match the alarm threshold as an abnormal performance indicator, and summarize the abnormal performance indicators according to the alarm level of the alarm rule to generate an abnormal performance indicator data set.

[0107] S40: issuing an alarm according to the abnormal performance indicator data set. The alarm is issued through the alarm channel of the alarm unit, and the alarm content includes the real-time performance indicator data and the abnormal performance indicator data set and the alarm level.

[0108] The present invention monitors the performance indicators of cloud computing resources in real time through the monitoring and alarm module, supports customized monitoring thresholds and alarm rules, and ensures the stability and reliability of the system. Through an effective alarm notification mechanism and data analysis, it helps the operation and maintenance team to identify and solve potential problems in a timely manner, thereby improving the utilization rate of cloud computing resources and the quality of service management.

[0109] Among them, the fault diagnosis module is configured to use machine learning and data analysis technology to intelligently diagnose and analyze the faults of the operation and maintenance management system based on cloud computing services, locate the root cause of the fault and give corresponding guidance and suggestions. Figure 5 As shown, the specific steps include:

[0110] S100, system data collection, including collecting log data, performance indicator data of cloud computing resources, system event data and historical fault data. Among them, collecting log data includes collecting system logs, application logs and security logs, etc., to obtain detailed information before and after the fault occurs. Collecting performance indicator data of cloud computing resources includes collecting performance indicator data of computing resources, storage resources and network resources, and constructing the environmental state when the fault occurs. Collecting system event data includes collecting system events, such as service start, stop, error code and abnormal events, etc., to help understand the context of the fault. Collecting historical fault data includes collecting historical fault cases and their solutions, providing a labeled data basis for subsequent model training.

[0111] S200, system data processing, including data preprocessing and feature extraction.

[0112] Data preprocessing includes data cleaning of collected log data, performance indicator data of cloud computing resources, system event data, and historical fault data, removing outliers, duplicate data, and noise from log data, performance indicator data of cloud computing resources, system event data, and historical fault data to ensure the quality of system data.

[0113] In one embodiment of the present invention, outliers, duplicate data and noise are removed from log data, performance indicator data of cloud computing resources, system event data and historical fault data, specifically including:

[0114] S201. Detect and remove outliers in system data. Outliers refer to values ​​that are significantly different from other data points. Detecting and removing outliers in system data includes:

[0115] Calculate the Z-score value and IQR value of each data in the log data, cloud computing resource performance indicator data, system event data, and historical fault data. The Z-score value is calculated using the Z-score method, and the IQR value is calculated using the interquartile range (IQR) method.

[0116] Identify and remove outliers based on the Z-score and IQR values ​​of each data point. Specifically:

[0117] Outliers are identified based on the Z-score value of each data. Usually, data with a Z-score value greater than 3 or less than -3 are considered the first outliers.

[0118] Outliers are identified based on the IQR value of each data. The threshold of the outlier is (Q1 - 1.5 *IQR, Q3 + 1.5*IQR). Data outside this range is marked as the second outlier.

[0119] Compare the first and second outlier values ​​of each data. When the first and second outlier values ​​are the same, the data is determined to be abnormal and removed. When the first and second outlier values ​​are different, calculate the Z-score and IQR value of each data again and make a secondary judgment. If the secondary judgment structure is still different, the data is determined to be non-abnormal and does not need to be removed.

[0120] The present invention detects double abnormal data by using the Z-score method and the IQR method, and removes the abnormal data when both detection methods are abnormal, thereby further improving the quality of the data and ensuring the accuracy and reliability of subsequent analysis.

[0121] S202: Remove duplicate data. Duplicate data may be caused by multiple collections or data merging. Removing duplicate data includes:

[0122] Complete duplicate row removal directly checks whether there are exactly the same rows in the data set and removes the duplicate rows.

[0123] Field-based duplicate check and removal checks duplicate records based on specific fields (such as timestamp, sensor ID, etc.) and keeps the latest or most relevant records. For example, for records with the same timestamp, you can choose to keep the latest record or the average.

[0124] S203, removing noise, where noise refers to random errors or unnecessary information in the data, which may affect the analysis results. In the embodiment of the present invention, commonly used denoising methods, such as moving average method and median filtering, can be used to remove noise.

[0125] Preferably, in some embodiments of the present invention, removing noise also includes processing missing values, which are missing parts in the data set and may affect the analysis results.

[0126] Through the above data preprocessing method, the present invention can effectively clean the fault monitoring data of the operation and maintenance management system, remove outliers, duplicate data and noise, thereby improving data quality and laying the foundation for subsequent data analysis and modeling.

[0127] Data preprocessing also includes integrating and normalizing the cleaned data to generate a unified data format.

[0128] Feature extraction includes feature selection and creation of new features. Features related to faults are selected through correlation analysis, principal component analysis (PCA), etc. to reduce redundant information. At the same time, new features are created, such as time series features and statistical features (mean, variance, etc.) to enhance the performance of the model.

[0129] S300, model training, including algorithm selection, model training and model evaluation. Among them, select appropriate machine learning algorithms according to data characteristics and problem types, such as decision tree, random forest, support vector machine (SVM), neural network, etc., which are not limited by the present invention; use labeled historical fault data to train the model, and adjust hyperparameters to optimize model performance. Use cross-validation and test sets to evaluate the accuracy, recall rate, F1-score and other indicators of the model to ensure the effectiveness of the model.

[0130] S400, fault detection, including real-time monitoring and anomaly detection. During system operation, the performance indicators, log data and system event data of cloud computing resources are monitored in real time, and the trained model is used for fault prediction. The model is used to analyze the performance indicators, log data and system event data of real-time cloud computing resources, identify abnormal patterns that are significantly different from normal behavior, and issue fault alarms in a timely manner.

[0131] S500, fault root location, including fault classification, root cause analysis and correlation analysis. Classify the faults detected in the fault detection step and determine the fault type (such as hardware fault, software fault, network fault, etc.); use decision tree, causal inference and other technologies to analyze the classified faults, analyze the causes of the faults, and locate the specific components or services; analyze the correlation between the fault type, fault cause and historical events, and the performance indicators of cloud computing resources to further confirm the root cause of the fault.

[0132] S600, generate fault solutions. Based on the fault type and root cause analysis results, generate corresponding fault solutions, such as restarting services, adjusting configurations, expanding resources, etc.; store the generated fault solutions in the database, and search for historical cases of similar fault types in the database for comparison and optimization, and propose successful optimal solutions; design an automated repair process based on the optimal solution, and the system automatically performs repair operations based on the optimal solution.

[0133] The present invention sets up a systematic process through the fault diagnosis module, combines machine learning and data analysis technology, and realizes intelligent diagnosis and analysis of system faults. This process can not only quickly locate the root cause of the fault, but also provide effective guidance and suggestions, thereby improving the reliability and flexibility of the system. Through continuous learning and optimization, the module can adapt to the ever-changing environment and improve the efficiency of fault handling.

[0134] The performance optimization module is configured to analyze and evaluate the performance of cloud computing resources, identify performance bottlenecks and optimization points, and provide performance optimization suggestions and solutions, such as resource adjustment, parameter configuration optimization, application architecture optimization, etc. Specifically, the performance of cloud computing resources is analyzed and evaluated, performance bottlenecks and optimization points are identified, and performance optimization suggestions and solutions are provided, including:

[0135] Configure monitoring tools (such as Prometheus, Grafana, New Relic, etc.), set monitoring indicators and thresholds, and collect performance data of cloud computing resources in real time, including CPU usage, memory usage, network traffic, response time, etc.

[0136] Generate performance reports of cloud computing resources regularly, and evaluate the performance trends of cloud computing resources based on historical data of cloud computing resources.

[0137] Design test scenarios, simulate different user behaviors, and generate performance test results for cloud computing resources. For example, conduct benchmark tests (such as using Apache JMeter, LoadRunner, etc.) to simulate user load and evaluate the performance of the operation and maintenance management system under high concurrency. Analyze the test results and identify interfaces with long response times or components with excessive resource utilization.

[0138] Based on the performance test results, identify deficiencies in parameter configuration, modify or adjust the parameter configuration of the application server, such as thread pool size, database connection pool size, etc., to improve the concurrent processing capability of the application; restart the service and monitor the performance changes of the modified or adjusted cloud computing resources.

[0139] Based on the performance changes of the modified or adjusted cloud computing resources, performance optimization suggestions and solutions are proposed, including redesigning the application architecture, separating functional modules into microservices, and using APIs for communication; configuring cache services and implementing data caching logic in the application; analyzing database query performance, creating or optimizing indexes based on query patterns, and monitoring query performance improvements.

[0140] The security management module is configured to manage and monitor the security of cloud computing resources, including user authentication, permission management, access control, security audit, etc. It also includes real-time monitoring of security events, such as intrusion detection, vulnerability scanning, virus protection, etc., and timely implementation of security protection measures. The security management module plays a vital role in the cloud computing environment, ensuring the security and compliance of cloud resources. Specifically, it includes:

[0141] User authentication uses multi-factor authentication (MFA) for user authentication, including: the user enters the user name and password on the login interface; the system verifies the user's credentials, the system sends a verification code to the user's mobile phone or email, and the user needs to enter the verification code to complete the identity authentication.

[0142] Permission management includes creating roles and permission models, and clarifying the scope of permissions for each role; for example, administrators can access all resources, developers can only access the development environment, and ordinary users can only access specific applications. When users register, they are assigned to the corresponding roles; when users access resources, the system performs the above-mentioned user authentication to check the permissions of the user role and decide whether to allow access.

[0143] Access control, implements resource-based access control, for example, restricting certain users to access only specific files or database tables. This includes: defining access policies for each resource to clarify which users or roles can access; when a user requests access to a resource, the system verifies based on the access policy to ensure that only authorized users can access it.

[0144] Security auditing records all user operations, including login, resource access, and permission modification, to ensure the traceability of system traces. For example, it records which resources a user logged in and accessed at a certain point in time. This includes: configuring the audit log function to ensure that the system can automatically record related events; regularly checking the audit log to identify abnormal behaviors, such as frequent failed login attempts or abnormal resource access.

[0145] Real-time monitoring of security events, including: deploying intrusion detection systems to monitor network traffic and system activities to identify potential intrusion behaviors, such as monitoring abnormal login attempts or unauthorized file access; regularly performing vulnerability scans on cloud resources to identify known vulnerabilities and fix them in a timely manner, such as using tools such as Nessus or OpenVAS to scan systems and find unpatched software; deploying antivirus software to monitor and isolate malware in real time. For example, using cloud antivirus services to automatically scan uploaded files and block malicious files.

[0146] Preferably, the security management module also includes configuring an automated response mechanism so that when a security incident is detected, the system can automatically take measures, such as blocking suspicious IP addresses or disabling user accounts.

[0147] The security management module of the present invention ensures the security of cloud resources by comprehensively implementing measures such as user authentication, authority management, access control, and security auditing. Real-time monitoring of security events and timely adoption of protective measures can effectively prevent and respond to potential security threats, further improving the reliability of the operation and maintenance management system.

[0148] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, the indirect coupling or communication connection of the modules can be electrical, mechanical or other forms.

[0149] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. An operation and maintenance management system based on cloud computing services, characterized in that: It includes resource management module, monitoring and alarm module, fault diagnosis module, performance optimization module and security management module for wireless communication connection; The resource management module is configured to uniformly manage cloud computing resources, including: Integrate and synchronize cloud computing resources. Use the APIs provided by various cloud services to regularly collect cloud computing resource data from different platforms to achieve automatic synchronization of cloud computing resource data. Establish a data warehouse to centrally store cloud computing resource data from different cloud services. Label and classify cloud computing resources, label different cloud computing resources stored in the data warehouse with unified labels, formulate unified classification standards, and classify cloud computing resource types by resource type, which are classified into: computing resources, storage resources, and network resources; Centralized management of tagged and classified cloud computing resource data, including resource creation, configuration, deletion, expansion and reduction operations; specifically, including: The user submits a resource creation request through the management interface or API, specifying the resource type, performance, and capacity parameters; the system allocates resources in the cloud platform according to the user's resource creation request, creates resources, and returns relevant resource creation information; The user configures the created resources and configures the corresponding resource permissions and parameter settings for the created resources; The user initiates a deletion request through the management interface or API and specifies the resources to be deleted; the system performs the deletion operation based on the received deletion request and releases the related resources and configurations; The user initiates a capacity expansion request through the management interface or API and specifies the resources to be expanded. The system configures the expansion of resources based on the expansion request. The user initiates a scaling request through the management interface or API and specifies the resources to be scaled down. The system scales down the resources according to the scaling request. The performance optimization module is configured to analyze and evaluate the performance of cloud computing resources, identify performance bottlenecks and optimization points, and provide performance optimization suggestions and solutions, including: Configure monitoring tools, set monitoring indicators and thresholds, and collect performance data of cloud computing resources in real time; Generate performance reports of cloud computing resources regularly, and evaluate the performance trends of cloud computing resources by combining historical data of cloud computing resources; Design test scenarios, simulate different user behaviors, and generate performance test results for cloud computing resources; Based on the performance test results, identify the deficiencies in parameter configuration, modify or adjust the parameter configuration of the application server; restart the service and monitor the performance changes of the modified or adjusted cloud computing resources; Propose performance optimization suggestions and solutions based on the performance changes of modified or adjusted cloud computing resources; The fault diagnosis module is configured to intelligently diagnose and analyze faults of the operation and maintenance management system based on cloud computing services based on machine learning and data analysis, locate the root cause of the fault and give corresponding guidance and suggestions; specifically, the steps include: S100, system data collection, including collecting log data, performance indicator data of cloud computing resources, system event data and historical fault data; wherein collecting log data includes collecting system logs, application logs and security logs to obtain detailed information before and after the fault occurs; collecting performance indicator data of cloud computing resources includes collecting performance indicator data of computing resources, storage resources and network resources; collecting system event data includes collecting system events, and the collected system events include service start, stop, error code and abnormal events; collecting historical fault data includes collecting historical fault cases and their solutions; S200, system data processing, including data preprocessing and feature extraction; Data preprocessing includes data cleaning of the collected log data, performance indicator data of cloud computing resources, system event data, and historical fault data, and removing outliers, duplicate data, and noise from the log data, performance indicator data of cloud computing resources, system event data, and historical fault data; The removal of outliers, duplicate data and noise from log data, performance indicator data of cloud computing resources, system event data and historical fault data specifically includes: S201. Detect and remove outliers in system data, where an outlier refers to a value that is significantly different from other data points. Detecting and removing outliers in system data includes: Calculate the Z-score value and IQR value of each data in the log data, the performance indicator data of the cloud computing resources, the system event data and the historical fault data; the Z-score value is calculated by the Z-score method, and the IQR value is calculated by the interquartile range method; Identify and remove outliers based on the Z-score and IQR values ​​of each data point; specifically: Identify outliers based on the Z-score value of each data, where data with a Z-score value greater than 3 or less than -3 are considered the first outliers; Identify outliers based on the IQR value of each data. The threshold of the outlier is (Q1 - 1.5 *IQR, Q3 + 1.5 *IQR). Data outside this range is marked as the second outlier. Compare the first outlier value and the second outlier value of each data. When the first outlier value is the same as the second outlier value, the data is determined to be abnormal and the abnormal data is removed. When the first outlier value is different from the second outlier value, calculate the Z-score value and IQR value of each data again and make a secondary judgment. If the secondary judgment results are still different, all data are determined to be non-abnormal and do not need to be removed. S202, removing duplicate data, including: Complete duplicate row removal, directly check whether there are exactly the same rows in the data set, and remove the duplicate rows; Field-based duplicate check and removal, check duplicate records based on specific fields and keep the latest or most relevant records; including: for records with the same timestamp, keep the latest record or the average value; S203, removing noise by using moving average method and median filtering; Data preprocessing also includes integrating and normalizing the cleaned data to generate a unified data format; Feature extraction is to select features related to faults through correlation analysis or principal component analysis, and create new features, including time series features, mean features and variance features; S300, model training, including: using annotated historical fault data to train decision trees, random forests, support vector machines or neural network models, adjusting hyperparameters to optimize model performance; using cross-validation and test sets to evaluate the accuracy, recall and F1-score indicators of the model; S400, fault detection, including: real-time monitoring of performance indicators, log data and system event data of cloud computing resources, and using the trained model to predict faults; analyzing the performance indicators, log data and system event data of real-time cloud computing resources through the model, identifying abnormal patterns that are significantly different from normal behaviors, and issuing fault alarms in a timely manner; S500, fault root location, including fault classification, root cause analysis and correlation analysis; classify the faults detected in the fault detection step and determine the fault type; use decision trees or causal inference logic to analyze the classified faults, analyze the causes of the faults, and locate the specific components or services; analyze the correlation between the fault type, fault cause, historical events, and performance indicators of cloud computing resources to further confirm the root cause of the fault; the fault types include hardware faults, software faults, and network faults; S600. Generate a corresponding fault solution based on the fault type and root cause analysis results; store the generated fault solution in a database, and search for historical cases of similar fault types in the database for comparison and optimization to extract the best successful solution; design an automated repair process based on the best solution, and the system automatically performs repair operations based on the best solution; the fault solution includes restarting services, adjusting configurations, and expanding resources.

2. The operation and maintenance management system based on cloud computing services according to claim 1, characterized in that: The monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue alarms in time when an abnormality occurs; the performance indicators of cloud computing resources include performance indicators of computing resources, performance indicators of storage resources and performance indicators of network resources.

3. The operation and maintenance management system based on cloud computing services according to claim 2, characterized in that: The monitoring alarm module includes a monitoring indicator unit, an alarm setting unit, a data analysis unit and an alarm notification unit connected to each other; wherein, The monitoring indicator unit is used to monitor the performance indicators of cloud computing resources in real time; The alarm setting unit is used to set the alarm threshold, alarm rules and alarm channels of the performance indicators; The data analysis unit is used to analyze the performance indicator data to determine whether to trigger the alarm condition of the alarm setting unit; The alarm notification unit is configured to trigger an alarm notification when the analysis result shows that the performance indicator reaches the alarm condition, and send the alarm notification through a preset channel.

4. The operation and maintenance management system based on cloud computing services according to claim 3, characterized in that: The monitoring alarm module is used to integrate alarm settings, integrating alarm thresholds, alarm rules and alarm channels into one alarm setting unit.

5. The operation and maintenance management system based on cloud computing services according to claim 4, characterized in that: The monitoring and alarm module is configured to monitor the performance indicators of cloud computing resources in real time and issue alarms in time when an abnormality occurs, including the following steps: S10, obtaining the type of cloud computing resources to be monitored; the cloud computing resource types include computing resources, storage resources and network resources; S20, obtaining real-time performance indicator data of the monitored cloud computing resources; extracting the real-time performance indicator data of the cloud computing resources using the monitoring indicator unit; S30, analyzing the acquired real-time performance indicator data to generate an abnormal performance indicator data set; S40. Issue an alarm based on the abnormal performance indicator data set.

6. The operation and maintenance management system based on cloud computing services according to claim 1, characterized in that: The security management module is configured to manage and monitor the security of cloud computing resources, including user authentication, authority management, access control, security auditing, and real-time monitoring of security events.

Images