Alarm rule automatic generation and closed loop optimization method and system

Abstract

The invention relates to an alarm rule automatic generation and closed loop optimization method and system, and relates to the field of computer system monitoring and testing. The method comprises the following steps: acquiring and cleaning a cluster to generate a standardized enhanced alarm event table based on multi-source heterogeneous data fusion; performing feature extraction on the enhanced alarm event table, constructing a feature vector, performing feature importance analysis on the feature vector by using a tree model, and outputting the feature importance analysis; according to statistical distribution of features in the structured feature vector, automatically generating a rule for identifying an alarm scene in combination with business knowledge; converting the alarm scene rule into a structured format, performing logic conversion through a test case generation engine, and outputting an automatic test case; and executing the automated test case and verifying the accuracy of the automated test case so as to dynamically optimize and adjust the alarm scene rule. According to the method, automatic rule generation and closed-loop optimization of a complex alarm scene can be realized, the alarm test efficiency and the scene recognition accuracy are remarkably improved, and the manual intervention requirement is reduced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer system monitoring and testing, and particularly relates to an alarm rule automatic generation and closed-loop optimization method and system. BACKGROUND

[0002] Alarm automatic testing, as an important part of system monitoring and operation and maintenance, is widely used in the fields of cloud computing, big data and distributed systems. With the continuous expansion of cluster scale and the continuous improvement of system complexity, alarm events show the characteristics of explosive growth in quantity, diversity in scene and strong correlation in context, and the traditional testing method has been difficult to meet the efficient and accurate testing demand. In related technologies, an alarm testing process is usually constructed through the coordinated work of data collection, rule configuration and testing execution. Specifically, the process covers the whole process from original alarm data acquisition, feature extraction, scene recognition to test case generation and verification, including key links such as multi-source heterogeneous data fusion, feature engineering construction and rule logic mapping. Among them, data processing and rule generation, as core supporting technologies, form a coordinated system with the testing execution system to jointly support the intelligent evolution of alarm testing.

[0003] However, in the existing alarm testing method, artificial simulation or static rule configuration is directly used, and the automatic generation and continuous optimization of rules are not realized, which may lead to problems such as incomplete test coverage, high rule maintenance cost, high false positive rate, or rule invalidation after system configuration changes, thereby affecting the testing efficiency and system stability. Specifically, the existing technology usually relies on artificial experience for rule writing, lacks systematic analysis of historical alarm data, and is difficult to identify features with high contribution to scene differentiation, and the generation and verification process of test cases lacks a feedback mechanism with the rule system, resulting in that the test results cannot drive the dynamic adjustment of rules. Based on this, the traditional method has significant limitations in dealing with complex and variable alarm scenes, and an automatic testing technology system with adaptability, interpretability and closed-loop optimization capability is urgently needed. SUMMARY

[0004] The present disclosure aims to at least partially solve one of the technical problems in the related art.

[0005] To this end, a first object of the present disclosure is to provide an alarm rule automatic generation and closed-loop optimization method.

[0006] A second object of the present disclosure is to provide an alarm rule automatic generation and closed-loop optimization system.

[0007] To achieve the above object, a first aspect of the present disclosure provides an alarm rule automatic generation and closed-loop optimization method, comprising:

[0008] S1, based on multi-source heterogeneous data fusion, obtaining and cleaning the historical alarm data, operation log, performance index, work order information and configuration change record of the cluster, generating a standardized enhanced alarm event table;

[0009] S2, feature extraction is performed on the enhanced alarm event table, a feature vector containing basic features, time sequence features and context features is constructed, and a tree model is used for feature importance analysis on the feature vector, and a sorted structured feature vector is output;

[0010] S3, according to the statistical distribution of the features in the structured feature vector, combined with business knowledge, automatically generating rules for identifying alarm scene;

[0011] S4, the alarm scene rules are converted into JSON Schema structured format, and logical conversion is performed through a test case generation engine, and an automatic test case is output;

[0012] S5, execute the automatic test case and verify its accuracy, and feed back the verification result to the alarm scene identification module, to dynamically optimize and adjust the alarm scene rules.

[0013] In an embodiment of the present application, the S1 comprises:

[0014] S11, by calling the API interface of the monitoring system, the log system, the performance index system, the work order management system and the configuration change management system, the historical alarm data, the operation log, the performance index, the work order information and the configuration change record of the cluster are collected regularly;

[0015] S12, standardizing the collected data, including unifying the timestamp to UTC format, unifying the enumeration value to standard format, and unifying the unit to standard unit.

[0016] In an embodiment of the present application, the S2 comprises:

[0017] S21, extracting basic features including alarm type, alarm source, alarm level, alarm occurrence time, duration, number of times, associated performance index value, associated configuration item and associated operation;

[0018] S22, using an XGBoost tree model to analyze the feature importance of the feature vector, sorting the features according to the feature weight, and outputting a structured feature vector containing feature names and importance scores.

[0019] In an embodiment of the present application, the S3 comprises:

[0020] S31, visualizing the value distribution of the Top-10 features under different alarm scenarios, including drawing a box plot, a histogram or a conditional probability diagram;

[0021] S32, extracting an interpretable logical condition from the Top-10 features using a shallow decision tree model as an identification rule of the alarm scenario.

[0022] In an embodiment of the present application, further comprising:

[0023] S6, deploying the automated test case to a test execution platform for simulation testing, recording the system response behavior in the test execution process, comparing the response behavior with the expected action, and generating a test verification report.

[0024] To achieve the above purpose, the second aspect of the present application provides an alarm rule automatic generation and closed-loop optimization system, comprising:

[0025] A data fusion and cleaning module is configured to fuse multi-source heterogeneous data, acquire and clean historical alarm data, operation logs, performance indicators, work order information and configuration change records of the cluster, and generate a standardized enhanced alarm event table;

[0026] A feature extraction and analysis module is configured to extract features from the enhanced alarm event table, construct a feature vector containing basic features, time series features and context features, and analyze the feature importance of the feature vector using a tree model, and output a sorted structured feature vector;

[0027] A rule automatic generation module is configured to automatically generate an alarm scenario rule according to the statistical distribution of the features in the structured feature vector, combined with business knowledge;

[0028] A rule structuring and test case generation module is configured to convert the alarm scenario rule into a JSON Schema structured format, and perform logical conversion through a test case generation engine to output an automated test case;

[0029] A test execution and feedback optimization module is configured to execute the automated test case and verify its accuracy, and feed back the verification result to the alarm scenario identification module to dynamically optimize and adjust the alarm scenario rule.

[0030] The method and device of the embodiments of the present application can realize automatic rule generation and closed-loop optimization of complex alarm scenarios, significantly improve alarm test efficiency and scenario recognition accuracy, and reduce manual intervention and maintenance cost.

[0031] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0032] The above and / or additional aspects and advantages of the present application will become apparent and be more readily understood through consideration of the following description, taken in conjunction with the accompanying drawings, in which:

[0033] Figure 1 is a flow chart of an alarm rule automatic generation and closed loop optimization method according to an embodiment of the present application;

[0034] Figure 2 is a whole logic diagram of an alarm rule automatic generation and closed loop optimization method according to an embodiment of the present application;

[0035] Figure 3 is a logic conversion schematic diagram of a test case generated by a test case generation engine according to an embodiment of the present application;

[0036] Figure 4 is a schematic diagram of verifying accuracy of an automation case according to an embodiment of the present application;

[0037] Figure 5 is a structure diagram of an alarm rule automatic generation and closed loop optimization system according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0039] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should belong to the protection scope of the present application.

[0040] A kind of alarm rule automatic generation and closed loop optimization method and system according to an embodiment of the present application will be described below with reference to the accompanying drawings.

[0041] Embodiment 1

[0042] Figure 1 is a flow chart of an alarm rule automatic generation and closed loop optimization method according to an embodiment of the present application, as shown in Figure 1 , comprising:

[0043] S1, based on multi-source heterogeneous data fusion, obtains and cleanses the historical alarm data, operation log, performance index, work order information and configuration change record of cluster, generates a standardized enhanced alarm event table.

[0044] Specifically, the step of "obtaining and cleaning the historical alarm data, operation logs, performance indicators, ticket information and configuration change records of the cluster based on multi-source heterogeneous data fusion, and generating a standardized enhanced alarm event table" is the core implementation link of the data processing module in the present application. The technical principle is based on data integration and feature engineering, aiming to provide high-quality, structured data input for subsequent alarm scenario recognition and automated testing.

[0045] In terms of technical implementation, this step first periodically collects multi-source heterogeneous data by calling the API interfaces of multiple systems in the cluster (such as monitoring systems, log systems, ticket management systems, configuration change systems, etc.). These data include but are not limited to historical alarm records, operation logs, performance indicators (such as CPU utilization, memory usage), ticket processing process and root cause analysis results, configuration change events, etc. Due to the diversity of data sources and the non-uniformity of formats, data cleaning and standardization processing are required. The cleaning process includes missing value filling (such as using mean or median filling), duplicate record identification and deletion, enumeration value unification (such as unifying "Error", "error", "ERR" to "ERROR"), timestamp format unification (such as converting to UTC time), and unit standardization (such as unifying to GB, MB, etc.).

[0046] Further, the cleaned data will be subjected to multi-dimensional association processing. For example, through the resource identifier (ResourceID), the alarm event is associated with the performance indicators, operation logs, ticket information and configuration change records within the time window (such as the performance data within 10 minutes before and after the alarm trigger). Finally, through the above data fusion and association operations, an enhanced alarm event table is generated, each record containing original alarm information, associated asset information, performance indicators, log content, ticket status and configuration change context, forming a multi-dimensional, structured alarm event data set.

[0047] In the parameter index layer, this step involves multiple key parameters, such as time window length (10 minutes), missing value filling strategy (mean filling), enumeration value mapping rule (case unification), unit conversion standard (such as storage unit unified to GB), feature extraction dimension (basic, time series, context feature), etc. In addition, the parameter settings of the XGBoost model (such as n_estimators = 100, max_depth = 5) also have a direct impact on the feature importance ranking results.

[0048] At the application scenario level, this step is widely applicable to alarm automation testing systems for large-scale distributed clusters, especially in complex IT environments such as cloud-native, microservices architecture, big data platforms, etc. Through the standardized enhanced alarm event table, the system can more accurately identify alarm scenarios, generate targeted test cases, and achieve closed-loop optimization, thereby improving the coverage and accuracy of alarm testing.

[0049] At the technical effect level, this step significantly improves the completeness and consistency of alarm data through multi-source data fusion and standardization, providing a high-quality data foundation for subsequent scenario recognition and test case generation. At the same time, through the introduction of feature engineering and tree model analysis, the interpretability and classification ability of alarm events are enhanced, providing key support for building an intelligent alarm testing system.

[0050] Further, S1 includes:

[0051] S1, based on multi-source heterogeneous data fusion, acquires and cleanses the historical alarm data, operation logs, performance indicators, ticket information, and configuration change records of the cluster, generating a standardized enhanced alarm event table, further including:

[0052] Specifically, this step involves historical alarm data acquisition and cleansing based on multi-source heterogeneous data fusion, aiming to build a standardized enhanced alarm event table to provide high-quality data input for subsequent alarm scenario recognition and automated testing. At the technical implementation level, this step first collects raw data through API interfaces of multiple systems in the cluster (such as monitoring systems, log systems, performance indicator systems, ticket management systems, and configuration change systems) on a regular basis, covering alarm logs, operation logs, performance indicators, ticket processing records, and configuration change events. The data collection frequency can be set to hourly, daily, or real-time synchronization according to actual business needs, ensuring the timeliness and completeness of the data.

[0053] In the data cleansing phase, the system uses various data preprocessing techniques, including missing value processing (such as using mean, median, or time series-based interpolation methods to fill in), duplicate record identification and deletion (through unique identifiers such as alarm IDs), format standardization (such as unifying timestamps to UTC format, enumerating values to uppercase form, and units to GB or MB, etc.). The cleaned data is associated across systems through resource identifiers (such as node IDs, service IDs), for example, matching alarm events with performance indicators, operation logs, ticket status, and configuration change records within a 10-minute time window, thereby constructing an enhanced alarm event table containing contextual information.

[0054] At the parameter index level, this step involves multiple key parameters, such as time window length (default 10 minutes), missing value filling strategy (optional mean, median or interpolation), enumeration value standardization rules (such as case conversion), unit uniform standard (such as memory unit unified as GB), feature extraction dimension (basic features, time series features, context features), etc. In addition, the feature importance analysis based on the XGBoost tree model uses the default parameters (such as `n_estimators = 100`, `max_depth = 5`) for training, and outputs the Top-10 features and their importance weights for subsequent rule generation.

[0055] At the application scenario level, this step is widely applicable to the alarm management and automated testing scenarios of large-scale distributed systems, such as cloud computing platforms, big data clusters, microservice architectures, etc. By building a standardized enhanced alarm event table, the system can effectively identify the feature patterns of different alarm scenarios and provide precise data support for the generation of automated test cases.

[0056] From the technical effect, this step significantly improves the availability and consistency of alarm data through multi-source data fusion and cleaning, laying a solid foundation for subsequent scenario recognition and test case generation. Its core value lies in realizing a data-driven alarm testing process, reducing manual intervention, improving test coverage and accuracy, and thus enhancing the intelligence level of the system's alarm response mechanism.

[0057] S11, by calling the API interfaces of the monitoring system, log system, performance indicator system, ticket management system and configuration change management system, regularly collects the historical alarm data, operation logs, performance indicators, ticket information and configuration change records of the cluster.

[0058] Specifically, this step involves regularly collecting the historical alarm data, operation logs, performance indicators, ticket information and configuration change records of the cluster by calling the API interfaces of multiple systems, which is the key input link of the data processing module in the entire alarm automation testing system. In some implementations, this step uses a distributed data collection architecture, periodically pulling data from monitoring systems (such as Prometheus, Zabbix), log systems (such as ELK Stack, Splunk), performance indicator systems (such as Grafana, InfluxDB), ticket management systems (such as Jira, ServiceNow) and configuration change management systems (such as Git, Ansible Tower) through timing tasks (such as scheduling mechanisms based on Cron or Airflow), with a collection frequency that can be set to every hour, every day or a custom time interval according to business needs.

[0059] At the technical implementation level, this step communicates with each system through RESTful API or gRPC interface, requesting parameters including time range (such as the last 7 days), resource identifier (such as node ID, service name), alarm type, etc. The response data is usually in JSON or XML format, which needs to be parsed and structured. At the parameter index level, the collected data needs to meet the timestamp accuracy (such as millisecond level), field integrity (such as alarm level, trigger time, duration, associated resources, etc.), and data consistency requirements (such as unified time format UTC, unit unified GB, MB, etc.). In addition, to ensure data quality, data verification mechanisms need to be set during the collection process, such as field missing rate below 5%, timestamp deviation less than 10 seconds, etc.

[0060] In application scenarios, this step is widely applicable to alarm testing environments of large-scale distributed systems, such as cloud computing platforms, microservice architectures, big data clusters, etc. The collected data will be used to build enhanced alarm event tables, providing high-quality data foundation for subsequent feature extraction, scenario identification, and test case generation. The technical value of this step lies in the realization of the automated integration of multi-source heterogeneous data, improving data acquisition efficiency and accuracy, and providing reliable data support for the intelligentization and automation of alarm testing.

[0061] S12, standardizing the collected data, including unifying the timestamp to UTC format, unifying the enumeration value to standard format, and unifying the unit to standard unit.

[0062] Specifically, this step involves standardizing the collected raw data, which is a key link in the data preprocessing phase of the alarm automation testing system. Its technical implementation principle is based on the standardization process of data cleaning and format unification, aiming to eliminate data heterogeneity and improve the accuracy and consistency of subsequent feature extraction and model training.

[0063] In terms of technical implementation, the standardization process includes three main sub-steps: timestamp unification, enumeration value standardization, and unit unification. First, the timestamp is unified into UTC format. By parsing the time field in the original data (such as ISO 8601, local time, etc.), and using the `pytz` or `datetime` library in Python for time zone conversion and format standardization, all time data is stored in the ISO 8601 format (such as `2024-04-05T14:30:00Z`) in the UTC+0 time zone, facilitating data alignment and time series analysis across systems and regions. Second, the enumeration value standardization uses regular expression matching and case conversion strategies to unify different forms of enumeration values such as "Error", "error", "ERR" into a predefined standard format (such as "ERROR"), to eliminate semantic ambiguity and improve the recognition accuracy of the classification model. Finally, unit unification is achieved through a mapping table, for example, converting non-standard units such as "MB", "mb", "megabytes" to the unified "GB", and performing numerical conversion (such as 1GB = 1024MB), to ensure that performance indicators are compared and analyzed in a unified dimension.

[0064] In application scenarios, this step is widely used in multi-source heterogeneous data integration scenarios, such as alarm data collected from different monitoring systems (such as Prometheus, Zabbix) or log systems (such as ELK, Splunk), which often have inconsistent formats. Through standardization processing, data can be ensured to be in a unified format for subsequent correlation analysis and feature extraction, providing high-quality input data for alarm scenario recognition.

[0065] The technical effect of this step is to significantly improve the consistency and processability of data, laying a foundation for subsequent feature engineering and model training. By eliminating format differences, data parsing errors can be reduced, system robustness can be improved, and interoperability between different data sources can be enhanced, thereby improving the overall efficiency and accuracy of alarm automation testing.

[0066] S2, feature extraction is performed on the enhanced alarm event table, a feature vector containing basic features, time series features, and context features is constructed, and a tree model is used to analyze the feature importance of the feature vector, outputting a sorted structured feature vector.

[0067] Specifically, this step involves feature extraction on the enhanced alarm event table and constructing feature vectors containing basic features, timing features, and context features, followed by feature importance analysis using tree models to output sorted structured feature vectors. This process is the core link of the data processing module in the entire alarm automation test system, and its technical implementation is based on multi-source heterogeneous data fusion and feature engineering methods, aiming to provide high-distinguishability and strong interpretability of feature input for subsequent alarm scenario recognition and test case generation.

[0068] In some implementations, the feature extraction process first extracts basic features from the enhanced alarm event table, such as alarm type (alert_type), alarm source (alert_source), alarm level (alert_level), alarm occurrence time (timestamp), duration (duration_sec), alarm count (count), CPU load at trigger time (cpu_util_at_trigger), memory utilization at trigger time (mem_util_at_trigger), etc. Timing features include alarm frequency (frequency), alarm interval (interval), number of triggers of the same alarm in the past 1 day (past_1d_same_alarm_count), and alarm sequence patterns (such as frequent occurrence of B alarm after A alarm). Context features include system overall load state (system_load), recent configuration change events (config_change), alarm status of related services (related_service_status), etc. These features are encoded and standardized through feature engineering methods, such as One-Hot encoding for enumeration values and time series feature extraction for timestamps (such as hours, days, holiday identifiers, etc.).

[0069] In feature importance analysis, an XGBoost (Extreme Gradient Boosting) tree model is used for training. The model parameters are set as: `n_estimators=100`, `max_depth=5`, `learning_rate=0.1`, `objective='multi:softmax'` (multi-classification task). The feature importance is obtained through the `feature_importances_` attribute of the model, which is based on weight-based importance scores, reflecting the frequency and contribution of features in the model splitting process. The sorted feature vector is output in JSON format, containing feature names (feature) and importance weights (importance), such as `{"feature":"duration_sec","importance":0.25}`, facilitating subsequent modules for rule extraction and test case generation.

[0070] This step is widely used in the alarm management scenarios of large-scale distributed systems, especially in complex systems such as cloud-native, microservices architecture, and big data platforms. By automatically extracting and analyzing alarm features, the system can identify high-impact alarm patterns, such as CPU overload, memory leakage, network interruption, and other typical scenarios. In actual deployment, this step can be integrated into the feature engineering pipeline of the alarm analysis platform as a pre-processing module for the rule engine, supporting dynamic rule generation and automatic test case writing.

[0071] This step introduces a tree model for feature importance analysis, significantly improving the accuracy and interpretability of feature selection. Compared with traditional manual experience-based feature selection, this method can automatically identify features with high discriminability for alarm scenarios based on historical data, thereby improving the accuracy of subsequent rule generation. In addition, the output of the structured feature vector in JSON format facilitates the connection with the test case template in JSON Schema format, providing a data foundation for realizing closed-loop optimization of alarm testing. This technical solution has significant value in improving alarm test coverage, reducing manual intervention, and enhancing the intelligent level of the system.

[0072] Further, S2 comprises:

[0073] S21, extracting basic features including alarm type, alarm source, alarm level, alarm occurrence time, duration, frequency, associated performance indicator value, associated configuration item, and associated operation.

[0074] Specifically, the step of "extracting basic features including alarm type, alarm source, alarm level, alarm occurrence time, duration, frequency, associated performance indicator values, associated configuration items, and associated operations" is one of the core steps of the data processing module in the present application. The technical implementation is based on the fusion of multi-source heterogeneous data and feature engineering methods, aiming to provide structured and high-discrimination feature inputs for subsequent alarm scenario recognition and automated test case generation.

[0075] At the technical implementation level, this step first obtains raw alarm event data from the cluster by calling API interfaces of monitoring systems, log systems, configuration management systems, etc. Then, the data is standardized and cleaned, including missing value filling (such as using mean or median), duplicate record deletion, timestamp format unification (such as UTC time), enumeration value unification (such as "Error", "error", "ERR" unified as "ERROR"), and unit standardization (such as unified as GB, seconds, etc.). After cleaning, the system associates the alarm event with its context data, including performance indicators (such as CPU utilization, memory usage), configuration change records (such as configuration item name, change time, change content), operation logs (such as user operation, command execution record), etc., forming an enhanced alarm event table.

[0076] At the parameter indicator level, the extraction of basic features needs to meet the following standards: the alarm type should be a predefined enumeration value (such as "CPU overload", "memory leak", "network interruption"); the alarm source should contain resource identifiers (such as node ID, service name); the alarm level should comply with the severity level in ISO / IEC 21823-3 standard (such as CRITICAL, MAJOR, MINOR); the alarm occurrence time and duration should be stored in ISO 8601 time format with a precision of seconds; the frequency statistics should be based on a time window (such as the number of times the same alarm is triggered in the past 24 hours); the associated performance indicator values should include numerical indicators (such as CPU utilization, disk IO) and their collection timestamps; the associated configuration items should include the configuration values before and after the change and the change time; the associated operations should record the operation type (such as "restart service", "update configuration") and the executing user.

[0077] At the application scenario level, this step is widely applicable to the alarm automated testing scenarios of large-scale distributed systems, especially in complex systems such as cloud-native, microservices architecture, big data platforms, etc. By extracting structured basic features, the system can identify the commonalities and differences of different alarm scenarios, providing accurate input data for the generation of automated test cases, thereby improving test coverage and accuracy.

[0078] At the technical effect level, this step significantly improves the explainability of alarm features and the quality of model input through multi-source data fusion and standardization. Feature importance ranking based on tree models such as XGBoost can effectively filter out features with high discriminative ability for alarm scenario recognition, providing data support for subsequent rule generation and test case design, and achieving intelligentization and automation of the alarm test process.

[0079] S22, performing feature importance analysis on the feature vector using an XGBoost tree model, ranking the features according to feature weights, and outputting a structured feature vector containing feature names and their importance scores.

[0080] Specifically, this step involves performing feature importance analysis on the feature vector using an XGBoost (eXtreme Gradient Boosting) tree model, ranking the features according to feature weights, and finally outputting a structured feature vector containing feature names and their importance scores. This step is a key link in feature engineering and model optimization in the entire alarm automation test system, and its technical implementation is based on the Gradient Boosting Decision Tree (GBDT) algorithm in ensemble learning, with high explainability and model performance.

[0081] At the technical implementation level, the XGBoost model performs ensemble learning by building multiple decision trees, and each tree optimizes the loss function through gradient descent to gradually improve the predictive ability of the model. In this invention, the input feature vector is generated by the data processing module and contains basic features (such as alarm type, source, level), time series features (such as alarm frequency, number of alarms within a time window), and context features (such as system load, configuration change records). During model training, XGBClassifier is used as the classifier, with n_estimators = 100 (i.e., 100 decision trees are constructed) and max_depth = 5 (the maximum depth of each tree is 5) to control the complexity of the model and prevent overfitting. After training, the feature_importances_ interface is called to obtain the weight scores of each feature, which are calculated based on the weighted average of the feature's split gain (Gain) in all decision trees.

[0082] At the parameter index level, the feature importance score is usually represented as a normalized value between 0 and 1, reflecting the contribution of the feature to the model's prediction. For example, the score of duration_sec is 0.25, indicating that it has high discriminative ability in the model. In addition, the feature ranking is based on the score from high to low, and the Top-10 features are selected as the input of the subsequent alarm scenario recognition module to improve the efficiency and accuracy of rule generation.

[0083] In the application scenario, this step is mainly used to filter out the most representative features from massive alarm data, providing high-discriminative data input for subsequent scene recognition and test case generation. For example, in identifying the "CPU overload" and "memory leakage" alarm scenarios, the feature importance analysis can quickly locate key indicators such as cpu_util_at_trigger, mem_util_at_trigger, etc., thereby constructing more targeted rules.

[0084] The technical effect of this step is to significantly improve the scientificity and automation of feature selection, reduce manual intervention, and improve model training efficiency and alarm recognition accuracy. By outputting a structured feature vector, the subsequent modules are provided with standardized and traceable data input, forming an end-to-end intelligent alarm management process. Further, this method can also combine with SHAP (SHapley Additive exPlanations) and other explanatory tools to enhance the explainability of feature importance, meeting the requirements of industrial-level systems for model transparency.

[0085] S3, according to the statistical distribution of Top-10 features in the structured feature vector, automatically generating rules for identifying alarm scene rules based on business knowledge.

[0086] Specifically, the step "according to the statistical distribution of Top-10 features in the structured feature vector, automatically generating rules for identifying alarm scene rules based on business knowledge" is one of the core technical implementations of the alarm scene recognition module in the alarm automation test system of the present application, and its technical principle is based on the rule generation mechanism combining feature importance analysis and statistical distribution modeling.

[0087] In terms of technical implementation, this step first relies on the structured feature vector output by the data processing module, where the Top-10 features are high-discriminative features extracted by XGBoost tree model (XGBClassifier) for feature importance sorting of alarm data. These features usually include alarm duration (duration_sec), CPU utilization at trigger (cpu_util_at_trigger), memory utilization at trigger (mem_util_at_trigger), same alarm count in the past day (past_1d_same_alarm_count), etc. In some implementations, the selection of Top-10 features can be based on feature importance weights (such as gain, weight, cover, etc.), and fine-tuned in combination with business experts' evaluation of feature explainability.

[0088] Further, the system statistically models the value distribution of the Top-10 features under different alarm scenarios. For example, using the Seaborn library to draw a violin plot or a histogram, the distribution difference of feature values under scenarios such as "CPU overload" and "memory leak" is analyzed. By calculating the conditional probability, the quartiles (Q1, Q2, Q3) of the box plot, and the outlier threshold (such as 1.5 times IQR), the system can identify feature intervals with significant distinguishing ability. Alternatively, a decision tree model (such as DecisionTreeClassifier, max_depth=3, min_samples_leaf=50) can also be introduced to logically divide the Top-10 features and extract rule conditions for scenario classification.

[0089] At the parameter index level, the feature importance ranking uses the default weight-based importance of XGBoost, and sets n_estimators=100 and max_depth=5 to ensure that the model has sufficient fitting ability. In statistical distribution analysis, the IQR calculation of the box plot, the bin width of the histogram, and the splitting conditions of the decision tree (such as gini index, information gain) all need to be dynamically adjusted according to the actual data distribution. In addition, during the rule generation process, the system supports setting a confidence threshold (such as p-value<0.05) to filter statistically insignificant feature conditions.

[0090] At the application scenario level, this step is widely applicable to alarm management scenarios of large-scale distributed systems, such as cloud computing platforms, microservice architectures, and big data clusters. By automatically identifying feature patterns of typical alarm scenarios such as "CPU overload", "memory leak", and "network delay", the system can provide executable alarm rule templates for operations personnel, which can be used for the generation and verification of automated test cases, thereby improving the accuracy and efficiency of alarm response.

[0091] The technical effect of this step is that it realizes the automatic generation of alarm scenario rules in a data-driven manner, significantly reducing the complexity and subjectivity of manually formulating rules, and improving the interpretability and generalization ability of the rules. At the same time, combined with the rule optimization mechanism of business knowledge, the generated rules have good adaptability and accuracy in the actual system, providing high-quality input for subsequent automated testing, and forming a closed-loop optimization intelligent alarm processing process.

[0092] Further, S3 includes:

[0093] S31, visualizes the value distribution of the Top-10 features under different alarm scenarios, including drawing a box plot, a histogram, or a conditional probability plot.

[0094] Specifically, this step involves visualizing the value distribution of the Top-10 features under different alarm scenarios, including drawing box plots, histograms, and conditional probability plots. The technical implementation principle is based on statistical analysis and data visualization techniques, aiming to reveal the distribution differences of key features in different alarm scenarios, thereby providing data support for subsequent alarm scenario recognition and rule generation.

[0095] At the technical implementation level, this step first extracts the values of the Top-10 features from the pre-processed alarm data set based on the XGBoost tree model output, and classifies them according to alarm scenarios (such as "CPU overload", "memory leak", "network interruption", etc.). Subsequently, using visualization libraries such as Seaborn and Matplotlib, the distribution of each feature under different scenarios is graphically displayed. For example, box plots are used to analyze the median, quartiles, and outliers of the feature, histograms are used to observe the frequency distribution of the feature values, and conditional probability plots are used to show the differences in conditional probability distribution of the feature under different scenarios. Through these charts, significant distinguishing points of features under different scenarios can be visually identified, providing a basis for subsequent threshold setting and rule extraction.

[0096] At the parameter index level, box plots usually set five-number summary (minimum, first quartile, median, third quartile, maximum) and outlier determination criteria (such as 1.5 times the interquartile range), and histograms need to set a reasonable bin size to avoid overfitting or information loss. Conditional probability plots can use kernel density estimation (KDE) or histogram overlay methods, combined with scenario labels for comparative analysis. In addition, when extracting rules, the decision tree model sets the maximum depth (max_depth = 3) and the minimum number of leaf node samples (min_samples_leaf = 50) to ensure that the generated rules have good generalization ability and interpretability.

[0097] At the application scenario level, this step is widely applicable to alarm automation test systems for large-scale cluster systems, especially in the multi-scenario, multi-dimensional alarm classification and rule formulation stage. By visualizing the distribution differences of Top features, the system can automatically identify feature combinations with high discrimination, thereby generating accurate alarm recognition rules and improving the coverage and accuracy of test cases.

[0098] The technical effect of this step is that the distribution of key features in different alarm scenarios is revealed in a data-driven manner, providing a quantitative basis for rule generation, significantly improving the automation level of alarm scenario recognition and the interpretability of rules, thereby enhancing the intelligent level and adaptability of the entire alarm automation test system.

[0099] S32, using a shallow decision tree model to extract interpretable logical conditions from the Top-10 features as the recognition rules of the alarm scenarios.

[0100] Specifically, this step involves using a shallow decision tree model (Decision Tree Classifier) to extract interpretable logical conditions from the Top-10 features as the recognition rules of the alarm scenarios. This step is the core part of the alarm scenario recognition module, and its technical implementation is based on the key features sorted by feature importance. By constructing a shallow decision tree model (max_depth=3, min_samples_leaf=50), logical rules with discriminative ability are extracted for automatic recognition and classification of different alarm scenarios.

[0101] In terms of technical implementation, this step first selects the top-10 most discriminative features based on the feature importance ranking output by the XGBoost model as the input variables of the decision tree. These features usually include alarm duration (duration_sec), CPU utilization at the time of triggering (cpu_util_at_trigger), memory utilization at the time of triggering (mem_util_at_trigger), the number of triggers of the same alarm in the past day (past_1d_same_alarm_count), operation log content (operate_log), etc. Then, the DecisionTreeClassifier class in the Scikit-learn library is used to build a decision tree model, with the maximum depth set to 3 (max_depth=3) and the minimum leaf node sample size set to 50 (min_samples_leaf=50) to ensure that the model structure is simple, the rule is highly interpretable, and overfitting is avoided.

[0102] In terms of parameter indicators, max_depth=3 limits the depth of the tree, so that the model only performs three rounds of feature partitioning, generating rules that are easy to understand and deploy; min_samples_leaf=50 ensures that each leaf node contains at least 50 samples, improving the generalization ability of the rules. In addition, the rules output by the model are usually exported in text form through the export_text function, such as "duration_sec>300 AND cpu_util_at_trigger>80%", which can be directly used to construct the alarm classification logic.

[0103] At the application scenario level, this step is widely used in the automatic testing of alarm in large-scale cluster systems. Through the extracted logical conditions, the system can automatically identify whether the alarm belongs to the typical scenarios such as "CPU overload", "memory leak", "configuration change exception", and generate corresponding test cases accordingly. For example, in a cloud platform monitoring system, this rule can be used to simulate the alarm triggering condition of specific hardware resource overload, to verify the accuracy and timeliness of the alarm response mechanism.

[0104] At the technical effect level, this step extracts the logical rules with business significance and interpretability from high-dimensional features through a shallow decision tree model, significantly improving the automation level of alarm scenario recognition and the maintainability of rules. Compared with the traditional manual rule configuration method, this method not only reduces manual intervention, but also improves the accuracy and adaptability of rules through data-driven methods, providing a solid foundation for the generation and verification of subsequent automatic test cases.

[0105] S4, convert the alarm scenario rules into JSON Schema structured format, and perform logical conversion through a test case generation engine to output automatic test cases.

[0106] Specifically, this step involves converting the alarm scenario rules into JSON Schema structured format, and performing logical conversion through a test case generation engine to finally output automatic test cases. The technical implementation principle is based on the structured modeling of rules and the automatic processing capability of the logic engine, aiming to improve the standardization, reusability and verifiability of test case generation.

[0107] At the technical implementation level, alarm scenario rules usually exist in natural language or semi-structured form, such as "trigger alarm when CPU temperature exceeds 85℃ and non-test environment". To realize automatic testing, such rules need to be converted into machine-readable and executable structured format. This invention uses JSON Schema as the rule modeling language, which has good semantic expression ability and type constraint mechanism, and can clearly define key elements such as alarm triggering conditions and expected response actions. For example, the alarm rule can be modeled as a JSON object containing `alert_id`, `trigger_conditions` (trigger condition array), `expected_actions` (expected action array) and other fields, where each trigger condition contains `metric` (metric name), `operator` (comparison operator), `threshold` (threshold) and `duration` (duration) and other attributes.

[0108] At the application scenario level, this step is widely applicable to alarm testing in complex cluster environments such as distributed systems, cloud computing platforms, and microservice architectures. For example, in a Kubernetes cluster, test cases covering various combinations of trigger conditions can be automatically generated for typical alarm scenarios such as node CPU overload, memory leakage, and network latency, to verify the accuracy of alarm rules and the integrity of system response mechanisms. Test cases can be integrated into CI / CD pipelines to enable continuous verification and iterative optimization of alarm rules.

[0109] At the technical effect level, this step enables structured representation of alarm rules and automatic generation of test cases, significantly improving test efficiency and coverage. Through standardized modeling with JSON Schema, different alarm rules can be uniformly managed and parsed, avoiding the inefficiency and errors of traditional manual test case writing. Meanwhile, the test case generation engine supports logical combination and condition nesting, enabling the generation of test cases that cover various boundary conditions, thereby enhancing the robustness and maintainability of the alarm system. Furthermore, test results can be fed back to the alarm scenario recognition module, forming a closed-loop optimization mechanism to continuously improve rule accuracy and system intelligence.

[0110] S5, executing the automated test cases and verifying their accuracy, feeding the verification results back to the alarm scenario recognition module to dynamically optimize and adjust the alarm scenario rules.

[0111] Specifically, this step "executing the automated test cases and verifying their accuracy, feeding the verification results back to the alarm scenario recognition module to dynamically optimize and adjust the alarm scenario rules" is a key closed-loop mechanism in the present application "Alarm automation testing method and implementation system", whose technical implementation principle is based on a test-driven rule optimization strategy. By automatically testing and verifying the accuracy of rules, the test results are fed back to the rule generation module to dynamically adjust and continuously optimize the rules.

[0112] At the technical implementation level, this step first calls the test case generation engine to convert the structured rules output by the alarm scenario recognition module (such as JSON Schema format) into executable test scripts. The test scripts simulate real alarm trigger conditions, such as setting CPU utilization above 85%, memory usage above 90%, and alarm duration above 5 minutes, and execute them in a test environment through simulators or test frameworks (such as JMeter, Selenium, custom alarm simulators). During test execution, the system records actual triggered alarm behaviors, including alarm response time, alarm level judgment, and whether triggered actions (such as notifications, automatic scaling, and log recording) meet expectations.

[0113] At the parameter index level, the verification process needs to evaluate multiple key indicators, including test case coverage (such as requiring more than 90% of the Top-10 feature combination scenarios), alarm identification accuracy (TPR > 95%), false positive rate (FPR < 5%), response delay (≤ 2 seconds), rule matching consistency (matching degree with historical alarm scene > 85%). The verification result is fed back to the alarm scene identification module in the form of structured data (such as JSON format), including test case ID, execution status (Pass / Fail), actual trigger condition, expected and actual response difference, etc.

[0114] At the application scene level, this step is widely used in alarm management of large-scale distributed systems, such as cloud computing platforms, micro-service architectures, and big data clusters. Through automated testing, the system can quickly verify its performance in a simulated environment after each rule update, ensuring that new rules have high robustness and accuracy in real scenarios, and avoiding false alarms or missed alarms caused by rule misjudgment.

[0115] At the technical effect level, this step realizes the "adaptive optimization" of alarm rules. Through the test feedback mechanism, the system can automatically identify rule defects and make adjustments, such as adding new conditions, modifying thresholds, and optimizing logic combination methods. This closed-loop mechanism significantly improves the intelligence level of the alarm system, reduces manual maintenance costs, and enhances the system's ability to identify complex alarm scenes. It is the core link of the end-to-end intelligent alarm management process.

[0116] The alarm rule automatic generation and closed-loop optimization method of the embodiment of the application realizes the automatic generation and verification of alarm test cases, improves the alarm test efficiency and rule accuracy, and reduces the need for manual intervention.

[0117] Further, it also includes:

[0118] S6, deploy the automated test case to the test execution platform for simulation testing, and record the system response behavior during the test execution process, compare the response behavior with the expected action, and generate a test verification report.

[0119] Specifically, this step involves deploying the automated test case to the test execution platform for simulation testing, recording the system response behavior, comparing it with the expected action, and finally generating a test verification report. This step is a key closed-loop link in the "alarm automation testing method and its implementation system" of the application, and its technical implementation is based on the structured expression of test cases, the execution capability of the test platform, and the automated processing of verification logic.

[0120] In terms of technical implementation, the automated test cases are first structured defined through JSON Schema format, including alert identification (alert_id), trigger conditions (trigger_conditions), and expected response actions (expected_actions). Among them, trigger_conditions is an array structure, and each element defines a specific performance indicator, operator (such as ">","<"), threshold value, and duration (such as "last for 5 minutes"), ensuring the executability and configurability of test conditions. The test case generation engine converts these rules into executable scripts or API call sequences recognized by the test platform, simulates alert trigger conditions, such as injecting specific performance indicators or log events, to activate the system alarm mechanism.

[0121] During the test execution process, the test platform needs to have multi-dimensional monitoring capabilities, including but not limited to alert trigger time, response delay, alert level, notification channel (such as email, SMS, ticket system), automatic scaling operation, log record integrity, etc. System response behavior will be collected and stored as structured logs or event streams in real time for subsequent comparison. The comparison logic is based on the preset expected action list (such as "notify", "auto_scale", "log", "ticket"), which verifies whether the actual response meets the expectations through state machines or rule engines, including whether the action occurs, the order of occurrence is correct, and the parameters match, etc.

[0122] At the parameter indicator level, the test platform needs to support concurrent testing (such as supporting up to 1000 concurrent test cases), response time precision (millisecond level), dynamic configuration of alert trigger conditions (supporting hot loading of JSON Schema format), storage format of test results (such as CSV, JSON, database table structure), etc. In addition, the verification logic needs to support fault tolerance mechanisms, such as allowing response delays not to exceed a set threshold (such as 500ms), or allowing the absence of some non-critical actions to not affect the overall verification result.

[0123] In application scenarios, this step is widely applicable to alarm verification of large-scale distributed systems, such as cloud computing platforms, microservice architectures, and big data clusters. By simulating real alarm scenarios, the accuracy of alarm rules, the integrity of alarm response mechanisms, and the reliability of automated processing flows can be verified, ensuring that the system has high availability and fault self-healing capabilities in actual operation.

[0124] The technical effect of this step is that, through automatic execution and verification, the test efficiency and coverage are significantly improved, manual intervention is reduced, and the consistency and traceability of test results are ensured. At the same time, the test verification report can be used as a feedback mechanism to drive the optimization of the alarm scene identification module rules, forming a closed-loop system of "rule generation → test verification → rule iteration", realizing the continuous evolution of alarm rules and the enhancement of system stability.

[0125] Embodiment 2

[0126] The present application proposes an alarm rule automatic generation and closed-loop optimization method, and the overall process of the scheme is as shown in Figure 2

[0127] In an embodiment of the present application, the data processing module mainly collects historical alarm data and associated data, and performs feature extraction. First, the module obtains historical data such as alarm logs, event logs, system indicators, application logs, configuration change records, etc. from the cluster. Then, the data is cleaned, missing values are processed, duplicate records or frequently occurring meaningless records are removed, and timestamps are standardized. Finally, the historical data related to the alarm are extracted, including the basic features, time sequence features and context features of the alarm, and the importance of the alarm features is analyzed and sorted based on the tree model, and a structured feature vector is output, providing data support for the alarm scene identification module.

[0128] Specifically, the data processing module cleans, correlates and extracts features from the original and discrete alarm events and related system data of the cluster, and finally converts them into structured and information-rich feature vectors for use by downstream models.

[0129] The alarm-related historical data is obtained from the cluster by calling the api interfaces of each system regularly, including obtaining the historical alarm records of the cluster from the monitoring system, obtaining the operation logs of the cluster from the log system, obtaining the real-time load of the cluster when the alarm occurs from the performance indicator monitoring system, obtaining the alarm handling process and root cause analysis results of the cluster from the ticket management system, and obtaining the cluster configuration change records from the configuration change management system.

[0130] Processing missing values: filling with mean. Processing duplicate values: identifying and deleting identical records. Format standardization: unifying timestamp format (such as UTC), unifying enumeration values (such as "Error", "error", "ERR" to "ERROR"), and unifying units (such as GB).

[0131] ​Performance indicator correlation: According to the resource identifier of the alarm source, the alarm trigger time point and the performance indicators of the alarm source or related nodes within 10 minutes before and after the alarm source are correlated to record the context state when the alarm occurs. Operation log correlation: According to the resource identifier of the alarm source, the error log and abnormal stack of the alarm source or related nodes are correlated to provide more detailed error information. Ticket event correlation: According to the alarm item ID identifier, it is correlated whether there is a processing ticket for the alarm, and the reason for the alarm at that time is recorded. Change correlation: According to the resource identifier of the alarm source, it is correlated whether there is a configuration change or code deployment of the alarm source or its dependent within a period of time before the alarm trigger, to identify the alarm caused by the change.

[0132] Through the correlation of alarm records and related data, an enhanced alarm event table is generated, and each record represents an alarm event, including original alarm information, associated asset information, performance indicator information at the correlation time, associated log information, associated ticket information, and associated configuration change information.

[0133] Further, the basic features include: alarm type, alarm source, level, alarm occurrence time, duration, number of times, associated performance indicator value (CPU load at the trigger time), associated configuration item, and associated operation. The time sequence features include: alarm frequency, interval, number of alarms in a specific time window, and alarm sequence pattern (such as frequent occurrence of B alarm after A alarm). The context features include: overall load state of the system at the time of alarm occurrence, recent deployment / configuration change event, and alarm state of associated other systems / services. The XGBoost tree model is used to analyze the importance of the alarm features and sort them, and a structured feature vector is output.

[0134] In an embodiment of the present application, the alarm scenario recognition module uses the structured feature vector obtained by the data processing module and the results of feature importance analysis to help develop and optimize rules for identifying specific alarm scenarios. First, according to the feature importance ranking, the top-10 important features are selected. Then, the value distribution of these important features under different alarm scenarios is analyzed (such as through box plots, histograms, conditional probabilities, etc. in the training data) to determine the threshold or conditions for distinguishing scenarios. Finally, combined with business knowledge and based on the values of important features, rules are automatically written for each target alarm scenario, so that different alarm scenarios correspond to specific alarm rules one by one.

[0135] Specifically, the alarm scenario recognition module uses the structured feature vector obtained by the data processing module and the results of feature importance analysis to help develop and optimize rules for identifying specific alarm scenarios. First, according to the feature importance ranking, the top-10 important features are selected. For each Top feature, a distribution plot is drawn according to historical alarms.

[0136] Then the distribution of these important features in different alarm scenarios is analyzed, and rules are extracted from the top features using a decision tree to determine thresholds or conditions for distinguishing scenarios.

[0137] In an embodiment of the application, the automated case generation and verification module automatically writes test cases and performs verification according to the alarm scenario rules output by the alarm scenario identification module. First, the alarm scenario rules are structured by using JSON Schema, and then the test case generation engine is used for logical conversion of the test cases to output corresponding automated test cases. Finally, the accuracy of the automated cases is verified and fed back to the alarm scenario identification module. The alarm identification module can continuously optimize and adjust the rules (such as adding / deleting conditions, adjusting thresholds) according to the results of the automated case verification.

[0138] Specifically, the automated case generation and verification module automatically writes test cases and performs verification according to the alarm scenario rules output by the alarm scenario identification module. First, the alarm scenario rules are structured by using JSON Schema. The test case generation engine is used for logical conversion of the test cases, as shown in Figure 3 The accuracy of the automated cases is verified, as shown in Figure 4 Finally, the results of the verification are fed back to the alarm scenario identification module, and the alarm identification module can continuously optimize and adjust the rules according to the results of the automated case verification.

[0139] In summary, the beneficial effects of the present application are as follows:

[0140] By integrating alarm logs, event logs, system indicators, application logs, configuration change records, and other multi-source heterogeneous data, combining data cleaning, timestamp normalization, and feature engineering (basic / sequential / contextual features), the tree model is used to automatically analyze feature importance and generate structured feature vectors, providing high-discrimination data input for scenario identification.

[0141] According to the feature importance analysis results, the statistical distribution (box plot, histogram, conditional probability) of key features is automatically analyzed, and interpretable rules (such as threshold or condition combination) are generated in combination with business knowledge, realizing dynamic matching of alarm scenarios and rules.

[0142] The alarm scenario rules are converted to JSON Schema structured format, and test cases are automatically generated using a test case generation engine, and the verification results are fed back to the rule module to form a closed-loop system of "rule generation → case testing → rule optimization".

[0143] A collaborative system composed of three modules of data processing, scene recognition, and use case generation / verification is designed, the rule dynamic adjustment is driven through feature importance analysis, and the rule effectiveness is verified through automatic test, forming an end-to-end intelligent alarm management process.

[0144] Embodiment 3

[0145] To achieve the above-mentioned embodiments, as Figure 5 shown, the embodiment also provides an alarm rule automatic generation and closed-loop optimization system 10, which includes:

[0146] A data fusion and cleaning module 100 is configured to fuse multi-source heterogeneous data, acquire and clean historical alarm data, operation logs, performance indicators, work order information, and configuration change records of a cluster, and generate a standardized enhanced alarm event table.

[0147] A feature extraction and analysis module 200 is configured to extract features from the enhanced alarm event table, construct a feature vector containing basic features, time series features, and context features, and perform feature importance analysis on the feature vector using a tree model, and output a sorted structured feature vector.

[0148] A rule automatic generation module 300 is configured to automatically generate alarm scene rules based on the statistical distribution of features in the structured feature vector, combined with business knowledge.

[0149] A rule structuring and test case generation module 400 is configured to convert the alarm scene rules into a JSONSchema structured format, and perform logical conversion through a test case generation engine, and output automatic test cases.

[0150] A test execution and feedback optimization module 500 is configured to execute the automatic test cases and verify their accuracy, and feed back the verification results to the alarm scene recognition module to dynamically optimize and adjust the alarm scene rules.

[0151] Further, the data fusion and cleaning module is further configured to:

[0152] By calling the API interfaces of the monitoring system, the log system, the performance indicator system, the work order management system, and the configuration change management system, the historical alarm data, the operation logs, the performance indicators, the work order information, and the configuration change records of the cluster are regularly collected.

[0153] The collected data is standardized, including unifying the time stamp to UTC format, unifying the enumeration value to standard format, and unifying the unit to standard unit.

[0154] Further, the feature extraction and analysis module is further configured to:

[0155] The extracted basic features include alarm types, alarm sources, alarm levels, alarm occurrence times, durations, times, associated performance indicator values, associated configuration items and associated operations.

[0156] The XGBoost tree model is used for feature importance analysis on the feature vector, the features are sorted according to the feature weights, and a structured feature vector containing feature names and importance scores is output.

[0157] Further, the rule automatic generation module is further used for:

[0158] The value distribution of the Top-10 features in different alarm scenarios is visually analyzed, including drawing a box plot, a histogram or a conditional probability diagram.

[0159] An interpretable logic condition is extracted from the Top-10 features using a shallow decision tree model, which is used as an identification rule for the alarm scenario.

[0160] Further, the system further comprises:

[0161] A test execution and report generation module is used to deploy the automatic test case to a test execution platform for simulation testing, record the system response behavior during test execution, compare the response behavior with the expected action, and generate a test verification report.

[0162] The alarm rule automatic generation and closed-loop optimization system of the embodiment of the application can realize automatic rule generation and closed-loop optimization of complex alarm scenarios, significantly improve the alarm test efficiency and scenario recognition accuracy, and reduce the need for manual intervention.

[0163] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the skilled person in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0164] Furthermore, the terms "first", "second", "third", "fourth", "fifth" and "sixth" are used herein for descriptive purposes only and are not to be construed as indicating or implying relative importance or a significant nature of so described technical features. It is to be understood that a technical feature described with the "first", "second", "third", "fourth", "fifth" or "sixth" can implicitly or explicitly include at least one of the technical features described with the "first", "second", "third", "fourth", "fifth" or "sixth". In the description of the present application, the meaning of "a plurality" is at least two, for example, two, three, etc., unless otherwise specifically defined.

Claims

1. A method for automatic generation and closed loop optimization of alarm rules, characterized in that, Comprise: S1, based on multi-source heterogeneous data fusion, obtain and clean the historical alarm data, operation log, performance index, work order information and configuration change record of the cluster, generate a standardized enhanced alarm event table; S2, feature extraction is carried out on the enhanced alarm event table, a feature vector containing basic features, time sequence features and context features is constructed, and feature importance analysis is carried out on the feature vector by using tree model, and a sorted structured feature vector is output; S3, according to the statistical distribution of the features in the structured feature vector, combined with business knowledge, automatically generate rules for identifying alarm scene; S4, convert the alarm scene rules into JSON Schema structured format, and perform logical conversion through test case generation engine, and output automatic test case; S5, execute the automatic test case and verify its accuracy, and feed back the verification result to the alarm scene recognition module, so as to dynamically optimize and adjust the alarm scene rules.

2. The method of claim 1, wherein, The S1 comprises: S11, by calling the API interface of the monitoring system, the log system, the performance index system, the work order management system and the configuration change management system, the historical alarm data, the operation log, the performance index, the work order information and the configuration change record of the cluster are collected regularly; S12, standardize the collected data, including unifying the timestamp to UTC format, unifying the enumeration value to standard format, and unifying the unit to standard unit.

3. The method of claim 1, wherein, The S2 comprises: S21, extract basic features including alarm type, alarm source, alarm level, alarm occurrence time, duration, number of times, associated performance index value, associated configuration item and associated operation; S22, use XGBoost tree model to analyze the importance of features in feature vector, sort features according to feature weight, and output structured feature vector containing feature name and importance score.

4. The method of claim 1, wherein, The S3 comprises: S31, visual analysis is performed on the value distribution of Top-10 features in different alarm scenes, including drawing box plot, histogram or conditional probability diagram; S32, use shallow decision tree model to extract interpretable logic conditions from Top-10 features as identification rules of alarm scene.

5. The method of claim 1, wherein, Further comprising: S6, deploy the automatic test case to the test execution platform for simulation test, record the system response behavior in the test execution process, compare the response behavior with the expected action, and generate test verification report.

6. An alarm rule auto-generation and closed loop optimization system, comprising: Comprise: A data fusion and cleaning module is used for obtaining and cleaning the historical alarm data, operation log, performance index, work order information and configuration change record of the cluster based on multi-source heterogeneous data fusion, and generating a standardized enhanced alarm event table; A feature extraction and analysis module is used for feature extraction on the enhanced alarm event table, constructing a feature vector containing basic features, time sequence features and context features, and performing feature importance analysis on the feature vector by using tree model, and outputting a sorted structured feature vector; A rule automatic generation module is used for automatically generating rules for identifying alarm scene according to the statistical distribution of features in the structured feature vector, combined with business knowledge; A rule structuring and test case generation module is configured to convert the alarm scenario rules into a JSON Schema structured format and perform logical conversion through a test case generation engine to output automated test cases. A test execution and feedback optimization module is configured to execute the automated test cases and verify their accuracy, and feed the verification results back to the alarm scenario identification module to dynamically optimize and adjust the alarm scenario rules.

7. The system of claim 6, wherein, The data fusion and cleaning module is further configured to: periodically collect historical alarm data, operation logs, performance indicators, ticket information, and configuration change records of the cluster by calling API interfaces of a monitoring system, a log system, a performance indicator system, a ticket management system, and a configuration change management system; perform standardized processing on the collected data, including unifying timestamps into UTC format, unifying enumeration values into a standard format, and unifying units into standard units.

8. The system of claim 6, wherein, The feature extraction and analysis module is further configured to: extract basic features including alarm types, alarm sources, alarm levels, alarm occurrence times, durations, frequencies, associated performance indicator values, associated configuration items, and associated operations; perform feature importance analysis on the feature vectors using an XGBoost tree model, sort the features according to their weights, and output structured feature vectors containing feature names and their importance scores.

9. The system of claim 6, wherein, The rule automatic generation module is further configured to: perform visual analysis on the value distribution of the Top-10 features in different alarm scenarios, including drawing box plots, histograms, or conditional probability plots; extract interpretable logical conditions from the Top-10 features using a shallow decision tree model as the identification rules of the alarm scenarios.

10. The system of claim 6, wherein, Further comprising: a test execution and report generation module configured to deploy the automated test cases to a test execution platform for simulation testing, record system response behaviors during the test execution process, compare the response behaviors with expected actions, and generate a test verification report.

Images