Alarm data classification method and system based on decision tree, and terminal

Abstract

The invention discloses an alarm data classification method, system and terminal based on a decision tree, and the method comprises the steps: obtaining operation and maintenance alarm data, carrying out the preprocessing and feature engineering extraction processing of the operation and maintenance alarm data, and obtaining alarm feature data; reprocessing the alarm feature data to obtain a training set and a verification set; determining a preset decision tree model, and performing model training processing and model optimization processing on the preset decision tree model according to the training set and the verification set to obtain an alarm classification model; and obtaining current alarm data, inputting the current alarm data into the alarm classification model, and outputting an alarm classification result. According to the method, the training set and the verification set are constructed by performing preprocessing and feature engineering processing on the operation and maintenance alarm data, and the alarm classification model is obtained by performing model training and optimization on the preset decision tree model, so that the alarm data can be quickly and accurately classified, and the classification efficiency of the alarm data is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of alarm processing, and in particular to an alarm data classification method, system, terminal and computer-readable storage medium based on a decision tree. Background Art

[0002] In the modern information technology environment, the operation and maintenance system faces a large amount of alarm information, which comes from various systems and devices, such as servers, network equipment, and applications.

[0003] Traditional alarm classification methods often rely on manual rules or simple thresholds for judgment, but the classification efficiency is low and the alarm classification results obtained are not accurate.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a decision tree-based alarm data classification method, system, terminal and computer-readable storage medium, aiming to solve the problem in the prior art that the classification efficiency of alarm information generated by the operation and maintenance system is low and the alarm classification results obtained are inaccurate.

[0006] To achieve the above object, the present invention provides a method for classifying alarm data based on a decision tree, and the method for classifying alarm data based on a decision tree comprises the following steps:

[0007] Acquire operation and maintenance alarm data, and perform preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data;

[0008] Reprocessing the alarm feature data to obtain a training set and a validation set;

[0009] Determine a preset decision tree model, and perform model training processing and model optimization processing on the preset decision tree model according to the training set and the validation set to obtain an alarm classification model;

[0010] The current alarm data is obtained, and the current alarm data is input into the alarm classification model, and the alarm classification result is output.

[0011] Optionally, the alarm data classification method based on decision tree, wherein the acquisition of operation and maintenance alarm data, and preprocessing and feature engineering extraction processing of the operation and maintenance alarm data to obtain alarm feature data, specifically includes:

[0012] Acquiring operation and maintenance alarm data, and preprocessing the operation and maintenance alarm data to obtain first operation and maintenance alarm data;

[0013] The operation and maintenance alarm data includes server system log data, network equipment monitoring data, and application error report data;

[0014] Wherein, the preprocessing includes data cleaning processing and data formatting processing;

[0015] Performing feature engineering extraction processing on the first operation and maintenance alarm data to obtain alarm feature data;

[0016] The alarm characteristic data includes time characteristics, alarm source characteristics and alarm description characteristics.

[0017] Optionally, the alarm data classification method based on decision tree, wherein the operation and maintenance alarm data is obtained, and the operation and maintenance alarm data is preprocessed and feature engineering extracted to obtain alarm feature data, and then further comprises:

[0018] The alarm feature data is subjected to classification and labeling processing to obtain a classification label corresponding to the alarm feature data.

[0019] Optionally, the decision tree-based alarm data classification method, wherein the reprocessing of the alarm feature data to obtain a training set and a validation set specifically includes:

[0020] Using a feature importance evaluation method to screen the alarm feature data to obtain first alarm extraction data;

[0021] Using the information gain method to perform data splitting and data pruning on the first alarm extraction data to obtain second alarm extraction data;

[0022] The cross-validation method is used to perform data partitioning processing on the second alarm extraction data to obtain a training set and a validation set.

[0023] Optionally, the alarm data classification method based on decision tree, wherein the step of determining a preset decision tree model and performing model training processing and model optimization processing on the preset decision tree model according to the training set and the validation set to obtain an alarm classification model, specifically includes:

[0024] Determine a preset decision tree model, and use a minimization loss function method to perform model training processing on the preset decision tree model according to the training set and the classification label to obtain an initial alarm classification model;

[0025] Obtaining an initial classification result obtained by inputting the validation set into the initial alarm classification model, and obtaining a performance indicator corresponding to the initial classification result, wherein the performance indicator includes an accuracy rate, a recall rate, and a harmonic mean of the accuracy rate and the recall rate;

[0026] The initial alarm classification model is optimized according to the performance index to obtain an alarm classification model.

[0027] Optionally, the alarm data classification method based on the decision tree, wherein the current alarm data is obtained, and the current alarm data is input into the alarm classification model, and the alarm classification result is output, is specifically:

[0028] Acquire current alarm data, and perform the preprocessing and feature engineering processing on the current alarm data to obtain current alarm feature data;

[0029] The current alarm feature data is input into the alarm classification model, and the alarm classification model is used to perform alarm classification processing on the current alarm feature data to obtain an alarm classification result.

[0030] Optionally, the alarm data classification method based on the decision tree, wherein the current alarm data is obtained, and the current alarm data is input into the alarm classification model, and the alarm classification result is output, and then the following is further included:

[0031] Determine a preset rule, and determine an alarm priority corresponding to the alarm classification result according to the preset rule, wherein the alarm priority includes a first alarm priority and a second alarm priority, and the first alarm priority is greater than the second alarm priority;

[0032] Obtain a first alarm processing method corresponding to the first alarm priority and a second alarm processing method corresponding to the second alarm priority;

[0033] When the alarm priority corresponding to the alarm classification result is the first alarm priority, the first alarm processing method is adopted for processing;

[0034] When the alarm priority corresponding to the alarm classification result is the second alarm priority, the second alarm processing method is adopted for processing.

[0035] In addition, to achieve the above-mentioned purpose, the present invention further provides an alarm data classification system based on a decision tree, wherein the alarm data classification system based on a decision tree comprises:

[0036] The alarm feature data acquisition module is used to acquire operation and maintenance alarm data, and perform preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data;

[0037] A data reprocessing module, used to reprocess the alarm feature data to obtain a training set and a verification set;

[0038] An alarm classification model generation module is used to determine a preset decision tree model, and perform model training processing and model optimization processing on the preset decision tree model according to the training set and the verification set to obtain an alarm classification model;

[0039] The alarm classification result output module is used to obtain current alarm data, input the current alarm data into the alarm classification model, and output the alarm classification result.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a decision tree-based alarm data classification program stored in the memory and executable on the processor, wherein the decision tree-based alarm data classification program, when executed by the processor, implements the steps of the decision tree-based alarm data classification method as described above.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a decision tree-based alarm data classification program, and when the decision tree-based alarm data classification program is executed by a processor, the steps of the decision tree-based alarm data classification method as described above are implemented.

[0042] In the present invention, operation and maintenance alarm data is obtained, and the operation and maintenance alarm data is preprocessed and feature engineering extracted to obtain alarm feature data; the alarm feature data is reprocessed to obtain a training set and a verification set; a preset decision tree model is determined, and the preset decision tree model is trained and optimized according to the training set and the verification set to obtain an alarm classification model; the current alarm data is obtained, and the current alarm data is input into the alarm classification model to output the alarm classification result. The present invention constructs a training set and a verification set by performing data preprocessing, feature engineering and reprocessing on the operation and maintenance alarm data, and adopts a decision tree algorithm to perform model training and model optimization on the decision tree model through the training set and the verification set to obtain an alarm classification model. The alarm classification model can quickly and accurately classify real-time alarm data, effectively improving the classification efficiency of alarm data, thereby helping operation and maintenance personnel to quickly handle alarm events and improve operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a preferred embodiment of the alarm data classification method based on decision tree of the present invention;

[0044] Figure 2 It is a data collection flow diagram of a preferred embodiment of the alarm data classification method based on a decision tree of the present invention;

[0045] Figure 3 It is a data preprocessing flow diagram of a preferred embodiment of the alarm data classification method based on a decision tree of the present invention;

[0046] Figure 4 It is a feature engineering flow chart of a preferred embodiment of the alarm data classification method based on decision tree of the present invention;

[0047] Figure 5 It is a flowchart of decision tree construction and training of a preferred embodiment of the alarm data classification method based on decision tree of the present invention;

[0048] Figure 6 It is a schematic diagram of an alarm classification process of a preferred embodiment of the alarm data classification method based on a decision tree of the present invention;

[0049] Figure 7 6. Alarm processing flow chart of a preferred embodiment of the alarm data classification method based on decision tree of the present invention;

[0050] Figure 8 It is a schematic diagram of a model update process of a preferred embodiment of the alarm data classification method based on a decision tree of the present invention;

[0051] Fig. 9 It is a structural diagram of a preferred embodiment of the alarm data classification system based on a decision tree of the present invention;

[0052] Fig.10 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] In the modern information technology environment, the operation and maintenance system is faced with a large amount of alarm information, which comes from various systems and devices, such as servers, network devices, and applications. Traditional alarm classification methods often rely on manual rules or simple threshold judgments, and have problems such as inaccurate classification, low efficiency, and inability to adapt to complex environmental changes. With the development of machine learning technology, the use of intelligent algorithms to achieve alarm classification has become a key direction to improve operation and maintenance efficiency and accuracy. As a classic machine learning algorithm, the decision tree algorithm has the advantages of strong interpretability and relatively high computational efficiency, and is suitable for the application scenario of alarm classification.

[0055] To solve the above problems, the present invention provides an alarm data classification method based on a decision tree, which can accurately and efficiently classify alarm information, effectively improve the efficiency of operation and maintenance personnel in handling alarms, reduce system failure time, and also improve the reliability and stability of the system.

[0056] The alarm data classification method based on decision tree described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the alarm data classification method based on the decision tree includes the following steps:

[0057] Step S10: Acquire operation and maintenance alarm data, and perform preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data.

[0058] The system architecture provided in the present invention includes a data acquisition module, a data preprocessing module, a feature engineering module, a decision tree construction and training module, an alarm classification module, an alarm processing module and a model updating module.

[0059] The first is the collection of operation and maintenance alarm data. In the operation and maintenance system, alarm information is collected from multiple data sources such as server system logs, network equipment monitoring data, and application error reports through the data acquisition module. These alarm information contains key information such as the time when the alarm occurred, the alarm source, and the alarm description.

[0060] Specifically, operation and maintenance alarm data is obtained, and the operation and maintenance alarm data is pre-processed to obtain first operation and maintenance alarm data; wherein the operation and maintenance alarm data includes server system log data, network equipment monitoring data and application error report data; wherein the pre-processing includes data cleaning processing and data formatting processing; feature engineering extraction processing is performed on the first operation and maintenance alarm data to obtain alarm feature data; wherein the alarm feature data includes time feature, alarm source feature and alarm description feature.

[0061] like Figure 2 As shown, in order to realize the collection of operation and maintenance alarm data, a data collection module is set in the present invention, and the data collection module obtains alarm information in a variety of ways. For server system logs, they can be read regularly through log collection tools; for network equipment monitoring data, they can be obtained through the SNMP protocol; for application error reports, they can be obtained through the application's own interface, and the collected data will be sent to the data preprocessing module.

[0062] like Figure 3As shown, after the operation and maintenance alarm data is obtained, the collected operation and maintenance alarm data can be preprocessed through the data preprocessing module, including data cleaning (removing duplicate alarm records, processing missing values ​​and correcting erroneous data) and data formatting (unifying the data into a format suitable for subsequent processing).

[0063] It is understandable that the data preprocessing module first cleans the collected alarm data and removes duplicate alarm records. If the same server sends the same alarm multiple times in a short period of time, only one record is retained; for missing values, a filling method is used, such as using the mean, median or mode to fill in numerical missing values, and using the most common category to fill in missing values ​​of categorical variables; for erroneous data, corrections are made, for example, if there are obvious spelling errors or non-standard expressions in the alarm description, corrections are made or the format is unified. Then, the cleaned alarm data is formatted and unified into a format suitable for subsequent processing, such as unifying the time format into a specific standard format, converting the alarm description into a vector form, etc.

[0064] Among them, data cleaning refers to removing duplicate alarm records, processing missing values ​​(such as using the mean, median or mode to fill in missing values ​​of numerical variables, and using the most common category to fill in missing values ​​of categorical variables), and correcting erroneous data (correcting spelling errors in alarm descriptions or unifying non-compliant expression formats).

[0065] Data formatting refers to unifying data into a format suitable for subsequent processing, such as unifying the time format into a specific standard format and converting the alarm description into a vector form (using TF-IDF processing).

[0066] like Figure 4 As shown, after the data preprocessing is completed, the preprocessed data is input into the feature engineering module. The feature engineering module is used to extract features from the preprocessed alarm data. The extracted features include time features (such as the hour, day, day of the week, etc. when the alarm occurs), alarm source features (such as the IP address of the alarm source converted to a network address segment, device type, etc.), and alarm description features (such as extracting key terms by performing bag-of-words model or TF-IDF processing on the alarm description).

[0067] Furthermore, for feature selection, the present invention uses a feature importance evaluation method (such as information gain) to screen the extracted features, exclude features that are irrelevant to the alarm type, such as the alarms reported by these devices are all error-level logs, then this feature is meaningless), and remove irrelevant or redundant features to reduce the complexity of the model and improve training efficiency. For example, the information gain of each feature is calculated, and features with larger information gain are selected to be retained, while those features that contribute less to classification are discarded.

[0068] Furthermore, the alarm feature data is subjected to classification and labeling processing to obtain a classification label corresponding to the alarm feature data.

[0069] After the data is extracted by the feature engineering model, the extracted features and corresponding classification labels (labeled by operation and maintenance experts) need to be sent to the decision tree construction and training module.

[0070] Step S20: reprocess the alarm feature data to obtain a training set and a validation set.

[0071] The present invention provides a decision tree construction and training module, which uses the extracted features and corresponding classification labels to construct a decision tree model. The decision tree is constructed using appropriate splitting criteria (such as information gain, gain ratio, etc.), and the model is trained and optimized by cross-validation and other methods to improve classification accuracy.

[0072] Specifically, a feature importance evaluation method is used to screen the alarm feature data to obtain first alarm extraction data; an information gain method is used to perform data splitting and data pruning on the first alarm extraction data to obtain second alarm extraction data; a cross-validation method is used to divide the second alarm extraction data to obtain a training set and a validation set.

[0073] For the data selection splitting criteria, it is necessary to select an appropriate splitting criterion to determine how to select the best splitting attribute at each node. Common splitting criteria include information gain (such as used in the ID3 algorithm) and gain ratio (such as used in the C4.5 algorithm). Taking information gain as an example, it measures the degree to which information uncertainty is reduced after a data set is split using a certain attribute. For each possible attribute, calculate its information gain and select the attribute with the largest information gain as the splitting attribute of the current node.

[0074] like Figure 5 As shown, the decision tree construction and training module in the present invention first screens the extracted features, uses information gain as a feature importance evaluation method, removes irrelevant or redundant preview features, reduces the complexity of the model, and improves training efficiency. Then, information gain is selected as the splitting criterion, the training data is split, and a decision tree is constructed. During the construction process, pruning operations are continuously performed to prevent overfitting. Finally, the training data is divided into several parts using a cross-validation method, and a part of it is used as a validation set each time, and the rest is used as a training set.

[0075] Step S30, determine a preset decision tree model, and perform model training processing and model optimization processing on the preset decision tree model according to the training set and the verification set to obtain an alarm classification model.

[0076] Specifically, a preset decision tree model is determined, and a loss minimization function method is used to perform model training processing on the preset decision tree model according to the training set and the classification label to obtain an initial alarm classification model; an initial classification result obtained by inputting the validation set into the initial alarm classification model is obtained, and performance indicators corresponding to the initial classification result are obtained, wherein the performance indicators include accuracy, recall rate, and the harmonic mean of the accuracy rate and the recall rate; model optimization processing is performed on the initial alarm classification model according to the performance indicators to obtain an alarm classification model.

[0077] The application of decision tree algorithm includes two aspects: feature selection and decision tree construction: 1. Feature selection: Before building the decision tree, the extracted features are screened by feature importance evaluation methods (such as information gain), irrelevant or redundant features are removed, the complexity of the model is reduced, and the training efficiency is improved. 2. Decision tree construction: Select appropriate splitting criteria (such as information gain) to split the training data and build a decision tree. During the construction process, pruning operations are continuously performed to prevent overfitting. Among them, the specific process of building a decision tree is as follows: Starting from the root node, the best splitting attribute is selected according to the selected splitting criteria to divide the data set into subsets. Each subset corresponds to a child node, and then this process is recursively repeated on each child node until the stopping condition is met. Stopping conditions: Common stopping conditions include the number of samples in the node is less than a certain threshold (such as stopping the splitting when the number of alarm records in the node is small to prevent overfitting), the samples in the node belong to the same category (that is, all alarms belong to the same category and there is no need to continue splitting) or the depth of the tree reaches the preset maximum value (to avoid the decision tree being too complex), etc.

[0078] For the training and optimization of the decision tree model: the present invention uses the cross-validation method to divide the training data into several parts, and uses a part of it as the validation set each time, and the rest as the training set. The model is trained on the training set by minimizing the loss function (such as the cross entropy loss function for classification tasks), and the model is optimized according to the performance indicators on the validation set (such as accuracy, recall, harmonic mean of accuracy and recall, etc.), and the trained decision tree model (i.e., the alarm classification model in the present invention) is sent to the alarm classification module.

[0079] It can be understood that the specific construction process of the decision tree is as follows: 1. Selection of splitting criteria: The present invention selects information gain as the splitting criterion to construct a decision tree. Information gain measures the contribution of a feature to the classification of a data set. By calculating the information gain of each feature, it can be determined which feature is selected at each node for splitting to achieve the best classification effect. 2. Calculate information gain: For each candidate feature, calculate its information gain. The calculation process is based on the information entropy of the data set, and then calculate the conditional entropy of the data set under the condition of a given feature. Finally, the information gain is calculated based on the information entropy and the conditional entropy. 3. Select the best splitting feature: According to the information gain of each feature calculated, select the feature with the largest information gain as the splitting feature of the current node. This feature divides the data set into multiple subsets, so that each subset has a purer category distribution on the feature, that is, most of the samples in each subset belong to the same category. 4. Generate child nodes: According to the selected splitting feature, divide the data set into corresponding subsets, and each subset corresponds to a child node of the decision tree. For example, if "alarm source feature (such as the conversion of the IP address of the alarm source to a network address segment)" is selected as the split feature, and the feature has different network address segment values, then child nodes will be generated, each of which corresponds to a sample subset of the network address segment value. 5. Recursive splitting: For each child node, repeat the above process of calculating information gain, selecting the best split feature, and generating child nodes until the stopping condition is met. The stopping condition can be that the samples in the node belong to the same category, or the number of samples in the node is less than a certain threshold, or the depth of the tree reaches the preset maximum value, etc. 6. Pruning operation: During the decision tree construction process, pruning operations are continuously performed to prevent overfitting. Pruning operations are divided into pre-pruning and post-pruning. Pre-pruning is to stop splitting in advance during the decision tree construction process. For example, when the number of samples of a node is too small or the information gain is less than a certain threshold, the node will no longer be split. Post-pruning is to prune the tree to simplify the model after the decision tree is built. A common post-pruning method is cost complexity pruning (also known as weakest link pruning). It defines a cost function, takes into account the complexity of the tree (such as the number of nodes or depth of the tree) and the classification error rate, starts from the leaf nodes, and gradually prunes upwards those subtrees that contribute the least to reducing the cost function. Post-pruning usually achieves better generalization performance than pre-pruning, but the computational overhead is relatively large because evaluation and pruning operations need to be performed on the entire constructed tree.

[0080] It is understandable that pre-pruning is used to prevent overfitting by stopping growth in advance during the decision tree construction process. For example, before the node splits, the validation set is used to evaluate the split of the current node. If the performance improvement after the split is not obvious (such as the accuracy is not significantly improved), the node is stopped from splitting. The advantage of pre-pruning is that the computational overhead is small, but it may cause underfitting because it stops the growth of the tree too early and may ignore some potentially useful splits. Post-pruning is to prune the decision tree after the decision tree is built. By evaluating the generalization performance of each subtree, those subtrees that do not improve the generalization performance or even have a negative impact are removed. Common post-pruning methods include cost complexity pruning, which calculates the cost complexity of each subtree and selects the subtree with the smallest cost complexity as the final decision tree.

[0081] Step S40: Acquire current alarm data, input the current alarm data into the alarm classification model, and output an alarm classification result.

[0082] The present invention also provides an alarm classification module: it is used to input the newly generated alarm data into a trained decision tree model (ie, the alarm classification model in the present invention) after feature extraction, and the model outputs the classification result of the alarm.

[0083] Specifically, current alarm data is obtained, and the preprocessing and feature engineering processing are performed on the current alarm data to obtain current alarm feature data; the current alarm feature data is input into the alarm classification model, and alarm classification processing is performed on the current alarm feature data through the alarm classification model to obtain an alarm classification result.

[0084] like Figure 6 As shown in the figure, the alarm classification module extracts the newly generated alarm data and inputs it into the trained decision tree model, and the model outputs the classification result of the alarm. For example, if the input alarm data meets the conditions of a branch in the decision tree model after feature extraction, then the alarm is classified into the corresponding category. Further, when the classification result is obtained, the classification result will be sent to the alarm processing module.

[0085] Further, a preset rule is determined, and the alarm priority corresponding to the alarm classification result is determined according to the preset rule, wherein the alarm priority includes a first alarm priority and a second alarm priority, and the first alarm priority is greater than the second alarm priority; a first alarm processing method corresponding to the first alarm priority and a second alarm processing method corresponding to the second alarm priority are obtained; when the alarm priority corresponding to the alarm classification result is the first alarm priority, the first alarm processing method is adopted for processing; when the alarm priority corresponding to the alarm classification result is the second alarm priority, the second alarm processing method is adopted for processing.

[0086] like Figure 7 As shown, an alarm processing module is also provided in the present invention. The alarm processing module determines the priority of the alarm according to the classification result, selects a suitable notification method (such as email, SMS or instant message, etc.) to send an alarm notification to the operation and maintenance personnel, and provides corresponding processing suggestions to guide the operation and maintenance personnel to quickly handle the alarm. For example, if the classification result is a hardware failure class, the priority is set to a high priority (i.e., the first alarm priority in the present invention). Select a suitable notification method according to the priority. For high-priority alarms, select a protocol method with strong immediacy such as SMS or instant messaging (i.e., the first alarm processing method in the present invention); for low-priority alarms (i.e., the second alarm priority in the present invention), select a notification method such as email (i.e., the second alarm processing method in the present invention). At the same time, according to the classification results and the operating experience of the system, corresponding processing suggestions are provided to the operation and maintenance personnel. For example, if the classification result is a hardware failure class, it is recommended to check the connection status and operating status of the relevant hardware equipment, and the notification method and processing suggestions are sent to the operation and maintenance personnel.

[0087] It can be understood that the implementation process of the alarm processing mechanism is as follows: 1. Priority determination: Determine the priority of the alarm based on the classification results of the alarm and the preset rules. For example, set high priority for hardware failure alarms and low priority for some general software prompt alarms. 2. Notification method selection: Select the appropriate telephone notification method based on the priority. For high-priority alarms, you may choose a more immediate notification method such as SMS or instant messaging; for low-priority alarms, you may choose email and other notification methods. 3. Providing processing suggestions: Based on the classification results of the alarm and the operating experience of the system, provide corresponding processing suggestions for the operation and maintenance personnel. For example, for hardware failure alarms, it is recommended to check the connection and operating status of the relevant hardware equipment; for software error alarms, provide corresponding software repair measures, etc.

[0088] like Figure 8As shown, the present invention also provides a model update module: for regularly collecting new alarm data, preprocessing it, feature engineering it, and retraining the decision tree model to adapt to changes in the system environment and alarm data patterns.

[0089] It can be understood that the model update module provided in the present invention is used to regularly collect new alarm data, perform preprocessing, feature engineering and retrain the decision tree model to adapt to changes in the system environment and alarm data patterns. The specific operations are similar to the above-mentioned data collection, preprocessing, feature engineering and decision tree construction and training steps, but the operations are performed on new alarm data.

[0090] Beneficial effects of the present invention:

[0091] 1. Improve classification accuracy: By adopting the decision tree algorithm, combined with feature engineering and optimization training methods, it is possible to accurately classify alarm information, reduce classification errors, and improve the accuracy of operation and maintenance personnel's judgment of system status.

[0092] 2. Improve operation and maintenance efficiency: Due to accurate alarm classification, operation and maintenance personnel can quickly locate and handle important alarms, reducing the time to find key information in a large amount of alarm information, thereby improving operation and maintenance efficiency.

[0093] 3. Adapt to system changes: By regularly updating the model, it can adapt to changes in the system environment and alarm data patterns, ensuring the continuity and stability of the classification effect.

[0094] Furthermore, if Fig. 9 As shown, based on the above-mentioned alarm data classification method based on decision tree, the present invention also provides an alarm data classification system based on decision tree, wherein the alarm data classification system based on decision tree includes:

[0095] The alarm feature data acquisition module 51 is used to acquire operation and maintenance alarm data, and perform preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data;

[0096] A data reprocessing module 52, used to reprocess the alarm feature data to obtain a training set and a verification set;

[0097] The alarm classification model generation module 53 is used to determine a preset decision tree model, and perform model training processing and model optimization processing on the preset decision tree model according to the training set and the verification set to obtain an alarm classification model;

[0098] The alarm classification result output module 54 is used to obtain current alarm data, input the current alarm data into the alarm classification model, and output the alarm classification result.

[0099] Furthermore, if Fig.10 As shown, based on the above-mentioned decision tree-based alarm data classification method and system, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Fig.10 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0100] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores an alarm data classification program 40 based on a decision tree, and the alarm data classification program 40 based on a decision tree can be executed by the processor 10, thereby realizing the alarm data classification method based on a decision tree in the present application.

[0101] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the decision tree-based alarm data classification method.

[0102] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0103] In one embodiment, when the processor 10 executes the decision tree-based alarm data classification program 40 in the memory 20 , the steps of the decision tree-based alarm data classification method described above are implemented.

[0104] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a decision tree-based alarm data classification program, and when the decision tree-based alarm data classification program is executed by a processor, the steps of the decision tree-based alarm data classification method as described above are implemented.

[0105] In summary, the present invention provides an alarm data classification method, system, terminal and computer-readable storage medium based on a decision tree, the method comprising: obtaining operation and maintenance alarm data, and performing preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data; reprocessing the alarm feature data to obtain a training set and a validation set; determining a preset decision tree model, and performing model training processing and model optimization processing on the preset decision tree model according to the training set and the validation set to obtain an alarm classification model; obtaining current alarm data, and inputting the current alarm data into the alarm classification model to output an alarm classification result. The present invention constructs a training set and a validation set by performing data preprocessing, feature engineering processing and reprocessing on the operation and maintenance alarm data, and adopts a decision tree algorithm to perform model training and model optimization on the decision tree model through the training set and the validation set, so as to obtain an alarm classification model. The alarm classification model can quickly and accurately classify real-time alarm data, effectively improving the efficiency of alarm data classification, thereby helping operation and maintenance personnel to quickly handle alarm events and improve operation and maintenance efficiency.

[0106] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0107] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0108] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A decision tree-based alarm data classification method, characterized in that: The alarm data classification method based on the decision tree includes: Acquire operation and maintenance alarm data, and perform preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data; Reprocessing the alarm feature data to obtain a training set and a validation set; Determine a preset decision tree model, and perform model training processing and model optimization processing on the preset decision tree model according to the training set and the validation set to obtain an alarm classification model; The current alarm data is obtained, and the current alarm data is input into the alarm classification model, and the alarm classification result is output.

2. The alarm data classification method based on decision tree according to claim 1 is characterized in that: The obtaining of operation and maintenance alarm data, and performing preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data specifically includes: Acquiring operation and maintenance alarm data, and preprocessing the operation and maintenance alarm data to obtain first operation and maintenance alarm data; The operation and maintenance alarm data includes server system log data, network equipment monitoring data, and application error report data; Wherein, the preprocessing includes data cleaning processing and data formatting processing; Performing feature engineering extraction processing on the first operation and maintenance alarm data to obtain alarm feature data; The alarm characteristic data includes time characteristics, alarm source characteristics and alarm description characteristics.

3. The alarm data classification method based on decision tree according to claim 1 is characterized in that: The operation and maintenance alarm data is obtained, and the operation and maintenance alarm data is preprocessed and feature engineering extracted to obtain alarm feature data, and then further includes: The alarm feature data is subjected to classification and labeling processing to obtain a classification label corresponding to the alarm feature data.

4. The alarm data classification method based on decision tree according to claim 1 is characterized in that: The reprocessing of the alarm feature data to obtain a training set and a validation set specifically includes: Using a feature importance evaluation method to screen the alarm feature data to obtain first alarm extraction data; Using the information gain method to perform data splitting and data pruning on the first alarm extraction data to obtain second alarm extraction data; The cross-validation method is used to perform data partitioning processing on the second alarm extraction data to obtain a training set and a validation set.

5. The method for classifying alarm data based on decision tree according to claim 3, characterized in that: The determining of a preset decision tree model, and performing model training processing and model optimization processing on the preset decision tree model according to the training set and the validation set to obtain an alarm classification model specifically includes: Determine a preset decision tree model, and use a minimization loss function method to perform model training processing on the preset decision tree model according to the training set and the classification label to obtain an initial alarm classification model; Obtaining an initial classification result obtained by inputting the validation set into the initial alarm classification model, and obtaining a performance indicator corresponding to the initial classification result, wherein the performance indicator includes an accuracy rate, a recall rate, and a harmonic mean of the accuracy rate and the recall rate; The initial alarm classification model is optimized according to the performance index to obtain an alarm classification model.

6. The method for classifying alarm data based on decision tree according to claim 1, characterized in that: The current alarm data is obtained, and the current alarm data is input into the alarm classification model, and the alarm classification result is output, specifically: Acquire current alarm data, and perform the preprocessing and feature engineering processing on the current alarm data to obtain current alarm feature data; The current alarm feature data is input into the alarm classification model, and the alarm classification model is used to perform alarm classification processing on the current alarm feature data to obtain an alarm classification result.

7. The method for classifying alarm data based on a decision tree according to claim 1, characterized in that: The method further comprises: obtaining current alarm data, inputting the current alarm data into the alarm classification model, and outputting an alarm classification result, and then further comprising: Determine a preset rule, and determine an alarm priority corresponding to the alarm classification result according to the preset rule, wherein the alarm priority includes a first alarm priority and a second alarm priority, and the first alarm priority is greater than the second alarm priority; Obtain a first alarm processing method corresponding to the first alarm priority and a second alarm processing method corresponding to the second alarm priority; When the alarm priority corresponding to the alarm classification result is the first alarm priority, the first alarm processing method is adopted for processing; When the alarm priority corresponding to the alarm classification result is the second alarm priority, the second alarm processing method is adopted for processing.

8. An alarm data classification system based on a decision tree, characterized in that: The decision tree-based alarm data classification system includes: The alarm feature data acquisition module is used to acquire operation and maintenance alarm data, and perform preprocessing and feature engineering extraction processing on the operation and maintenance alarm data to obtain alarm feature data; A data reprocessing module, used to reprocess the alarm feature data to obtain a training set and a verification set; An alarm classification model generation module is used to determine a preset decision tree model, and perform model training processing and model optimization processing on the preset decision tree model according to the training set and the verification set to obtain an alarm classification model; The alarm classification result output module is used to obtain current alarm data, input the current alarm data into the alarm classification model, and output the alarm classification result.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a decision tree-based alarm data classification program stored in the memory and executable on the processor. When the decision tree-based alarm data classification program is executed by the processor, the steps of the decision tree-based alarm data classification method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a decision tree-based alarm data classification program, and when the decision tree-based alarm data classification program is executed by a processor, the steps of the decision tree-based alarm data classification method according to any one of claims 1 to 7 are implemented.

Images