Alarm classification method, device, electronic equipment and storage medium
By extracting effective semantic words and feature vectors of network equipment alarm data, combined with classification model, the problems of complexity and accuracy of alarm classification in the financial industry are solved, efficient alarm classification is achieved, and manual maintenance costs and resource waste are reduced.
Patent Information
- Application Number
- CN201910482036.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-06-04
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-06-04
AI Technical Summary
In the prior art, the classification of alarms in the financial industry is complex and inaccurate, requiring a lot of manpower to configure and maintain, and reconfiguration is required when new equipment or software versions are replaced, resulting in waste of resources and increased costs.
By obtaining the alarm data of network equipment, extracting effective semantics and determining the alarm feature vector, the pre-trained classification model is used to determine the alarm category based on the conditional probability and estimated probability of historical alarm data, reducing manual labeling costs and improving classification accuracy.
It realizes accurate feature extraction and classification of complex alarm content, reduces manual configuration and maintenance costs, and improves the accuracy and efficiency of alarm classification.
Smart Images

Figure CN110287316B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of fintech, and in particular to an alarm classification method, device, electronic device and storage medium. Background Art
[0002] With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming into Fintech, and alarm classification technology is no exception. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on the technology.
[0003] For network operation and maintenance in the financial industry, existing technical solutions all implement classification of alarm information by having operation and maintenance personnel configure alarm keywords.
[0004] Each category has a corresponding keyword. Keywords are strictly used to match alarm information. Large financial companies with multiple IDCs often need network equipment from multiple vendors to avoid risks. After the equipment is put on the shelves, due to the complexity brought by multiple vendors, the specific alarm classification needs to consist of the following steps: first, put the equipment on the shelves, check whether the new equipment is a new brand model, check whether the alarm keyword of the brand model has changed, and finally configure the alarm keyword as needed.
[0005] Therefore, alarm classification requires the investment of O&M development manpower, and the alarm configuration system requires long-term maintenance and updates from O&M development manpower. New manufacturers, new models, new software versions, and even changes in network management system suppliers may all require reconfiguration, which is a huge waste of resources. Summary of the invention
[0006] The embodiments of the present invention provide an alarm classification method, device, electronic device and storage medium to solve the problem of accurate and complex content classification in the prior art.
[0007] The specific technical solutions provided by the embodiments of the present invention are as follows:
[0008] An embodiment of the present invention provides an alarm classification method, including:
[0009] Obtain alarm data generated by network devices;
[0010] Extracting valid semantic words from the alarm data, and determining an alarm feature vector corresponding to the alarm data according to the valid semantic words of the alarm data; wherein each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data;
[0011] The alarm category of the alarm data is determined according to the alarm feature vector of the alarm data and a pre-trained classification model; the classification model is determined according to the estimated probability of the historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of the historical alarm data under each alarm category is determined according to the conditional probability of each characteristic element of the historical alarm data in each alarm category.
[0012] In a possible implementation, the classification model is determined based on the estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data, including:
[0013] Acquire a historical alarm data set of the network device; the historical alarm data set includes a plurality of historical alarm data and an alarm category corresponding to each historical alarm data;
[0014] For each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data;
[0015] For the same alarm category, determining the conditional probability of each characteristic element in the alarm category;
[0016] For each historical alarm data, determining the estimated probability of the alarm feature vector of the historical alarm data under each alarm category according to the conditional probability of each characteristic element in the alarm feature vector of the historical alarm data under each alarm category;
[0017] The classification model is trained according to the estimated probability of each historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data.
[0018] In a possible implementation, the training of the classification model according to the estimated probability of each historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data includes:
[0019] Determine the prediction classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category;
[0020] According to the predicted classification and the alarm category corresponding to each historical alarm data, the parameters of the classification model are adjusted so that the confidence of the predicted classification is greater than a preset threshold.
[0021] In a possible implementation, for each historical alarm data, the estimated probability of the alarm feature vector of the historical alarm data under each alarm category is determined according to the conditional probability of each characteristic element in the alarm feature vector of the historical alarm data under each alarm category, which is determined according to the following formula:
[0022]
[0023] Where x = {a1, a2, ..., a m} is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the ith alarm category in the historical alarm data set; P(a j |y i ) is the first j The conditional probability of a feature element in the i-th alarm category; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
[0024] In a possible implementation manner, the method further includes:
[0025] If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
[0026] Another embodiment of the present invention provides an alarm classification device, comprising:
[0027] A transceiver unit, used to obtain alarm data generated by network equipment;
[0028] A processing unit, used for extracting valid semantic words from the alarm data, and determining an alarm feature vector corresponding to the alarm data based on the valid semantic words of the alarm data; wherein each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data; determining an alarm category of the alarm data based on the alarm feature vector of the alarm data and a pre-trained classification model; the classification model is determined based on an estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of the historical alarm data under each alarm category is determined based on the conditional probability of each feature element of the historical alarm data in each alarm category.
[0029] In a possible implementation manner, the transceiver unit is further used to:
[0030] Acquire a historical alarm data set of the network device; the historical alarm data set includes a plurality of historical alarm data and an alarm category corresponding to each historical alarm data;
[0031] The processing unit is also used to: for each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data; for the same alarm category, determine the conditional probability of each feature element in the alarm category; for each historical alarm data, determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category according to the conditional probability of each feature element in the alarm feature vector of the historical alarm data in each alarm category; and train the classification model according to the estimated probability of each historical alarm data in each alarm category and the alarm category corresponding to each historical alarm data.
[0032] In a possible implementation manner, the processing unit is specifically configured to:
[0033] Determine the predicted classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category; adjust the parameters of the classification model according to the predicted classification and the alarm category corresponding to each historical alarm data so that the confidence of the predicted classification is greater than a preset threshold.
[0034] In a possible implementation, the processing unit is specifically configured to determine the estimated probability of the alarm feature vector of the historical alarm data under each alarm category according to the following formula:
[0035]
[0036] Where x = {a1, a2, ..., a m} is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the ith alarm category in the historical alarm data set; P(a j |y i ) is the first j The conditional probability of a feature element in the i-th alarm category; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
[0037] In a possible implementation manner, the processing unit is further configured to:
[0038] If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
[0039] Another embodiment of the present invention provides an electronic device, including:
[0040] at least one memory for storing program instructions;
[0041] At least one processor is used to call the program instructions stored in the memory and execute any one of the above alarm classification methods according to the obtained program instructions.
[0042] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned alarm classification methods when executed by a processor.
[0043] In an embodiment of the present invention, an alarm feature vector of an alarm is calculated based on the alarm data of a network device. For alarms of complex content types, the features of the alarm can be more accurately represented, thereby solving the problem of difficulty in extracting features of alarms of complex content. The category of the alarm is determined based on the alarm feature vector and a classification model, which can greatly improve the accuracy of alarm classification. Furthermore, the alarm feature vector determined based on the alarm data of the network device is related to the alarm category. Therefore, it is not necessary to know a large number of training samples in advance during the training process, thereby reducing the cost of manually labeling the alarm categories in advance and reducing the complexity. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 Schematic diagram of an application scenario of the alarm classification method in an embodiment of the present invention;
[0045] Figure 2 is a flow chart of an alarm classification method in an embodiment of the present invention;
[0046] Figure 3 A flowchart of an alarm classification method according to an embodiment of the present invention;
[0047] Figure 4 This is a schematic diagram of the structure of an alarm classification device in an embodiment of the present invention;
[0048] Figure 5 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] To facilitate understanding of the embodiments of the present invention, several concepts are briefly introduced below:
[0051] Log alarm data: Alarm data extracted from logs generated by network devices in chronological order.
[0052] Bayesian classifier: Naive Bayesian classifier is a commonly used classifier in NLP. From a mathematical point of view, the classification problem can be defined as follows:
[0053] Known set: C = {y1, y2, ..., y n} and I = {x1, x2, ..., x m , ...}, determine the mapping rule y = f(x), so that any x i ∈I has only one y j ∈C such that y j =f(x i ) was established.
[0054] Where C is called the category set, each element of which is a category, and I is called the item set, each element of which is an item to be classified, and f is called the classifier. The task of the classification algorithm is to construct the classifier f.
[0055] The basis of Bayesian classifier is Bayesian theorem. This theorem solves the problem of how to get the probability of two events being exchanged when a certain conditional probability is known, that is, how to obtain P(B|A) when P(A|B) is known. Specifically, P(A|B) represents the probability of event A occurring under the premise that event B has already occurred, which is called the conditional probability of event A under the occurrence of event B. The basic solution formula is:
[0056]
[0057] Since P(A|B) can be easily and directly derived, while P(B|A) is difficult to directly derive, but we are usually more concerned with P(B|A), Bayes' theorem provides a method to derive P(B|A) from P(A|B). That is,
[0058]
[0059] refer to Figure 1As shown, it is a schematic diagram of the application scenario of the alarm classification method in the embodiment of the present invention. It includes network equipment and a network management system, wherein the network equipment generates corresponding logs during the network operation process, for example, in the process of providing services to user equipment, and the network management system can extract the alarm data in the log through the log of the network equipment, and then determine the classification of the alarm data, so as to facilitate the classification and processing of the alarm. Among them, the network equipment can be any device that can provide Internet services; the user equipment can be any smart device such as a smart phone, a tablet computer, a portable personal computer, a smart TV, etc., and the network management system can be a network management device associated with the network equipment.
[0060] The network devices and the network management system are connected via the Internet to achieve mutual communication. Optionally, the Internet uses standard communication technologies and / or protocols. The Internet is usually the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.
[0061] It should be noted that the alarm classification method provided in the embodiment of the present invention may be implemented by a network management system, or by an alarm classification device provided in a network device, or by a separate alarm classification device, which is not limited here.
[0062] In various embodiments of the present invention, the alarm classification method is used to Figure 1 It is worth noting that the application scenario architecture diagram in the embodiment of the present invention is to more clearly illustrate the technical solution in the embodiment of the present invention, and does not constitute a limitation on the technical solution provided in the embodiment of the present invention. For other application scenario architectures and business applications, the technical solution provided in the embodiment of the present invention is also applicable to similar problems.
[0063] For the network operation and maintenance of the financial industry, such as the network operation and maintenance of financial institutions such as banks, the existing technical solutions are to implement the classification of alarm information by configuring alarm keywords by operation and maintenance personnel. Each classification has a corresponding keyword. Strictly use keywords to match alarm information. If a large financial company sets up multiple IDCs, in order to avoid risks, it needs network equipment from multiple manufacturers. After the equipment is put on the shelves, due to the complexity brought by multiple manufacturers, the specific alarm classification needs to first perform the equipment on-shelf operation, check whether the new equipment is a new brand model, check whether the alarm keywords of the brand model have changed, and finally configure the alarm keywords as needed. In this way, relying solely on the pre-set alarm keywords is far from enough to extract the content features of the alarm, which can easily cause inaccurate classification. At present, for the different types of alarm data generated by the logs generated by these different network devices, keywords need to be set separately. Alarm classification requires the investment of operation and maintenance development manpower, and the alarm configuration system requires long-term maintenance and update by the operation and maintenance development manpower. New manufacturers, new models, new software versions, and even the replacement of the network management system supplier may all need to be reconfigured, which greatly increases the cost of operation and maintenance.
[0064] Based on the above issues, see Figure 2 FIG. 1 is a flowchart of an alarm classification method according to an embodiment of the present invention, and the method includes:
[0065] Step 201: Acquire alarm data generated by network equipment;
[0066] Specifically, I = {x1, x2, ..., x m , ...} can be expressed as an alarm data set, x1, ...x m The alarm data is captured from the logs of network devices.
[0067] Step 202: extracting valid semantic words from the alarm data, and determining an alarm feature vector corresponding to the alarm data according to the valid semantic words of the alarm data;
[0068] Wherein, each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data;
[0069] Specifically, in this scenario, valid sentences of the alarm data can be first determined as further extraction of the alarm information, and then the valid sentences can be segmented to extract valid semantic words in the alarm data. Among them, the valid semantic words can be words related to the alarm classification, or they can be determined by determining words that are not related to the alarm classification. In the specific implementation process, there can be multiple methods for extracting valid semantic words in the alarm data.
[0070] A possible implementation method is to remove words irrelevant to alarm classification, such as time and prepositions in the sentence, from the valid sentence, and use the remaining words as valid semantic words.
[0071] For example, a valid sentence in the alarm data is: VPN xxx is disconnected. It can be determined that the word "is" is irrelevant to the alarm classification. Furthermore, it can be determined that the valid semantic words are "VPN", "xxx", and "disconnected".
[0072] In another embodiment, considering the high real-time requirements in the actual production environment, the loop through which the data needs to pass needs to be as short as possible to improve effectiveness. The time spent on extracting the effective semantic words in the alarm data is not conducive to improving effectiveness, especially when the effective sentences in the alarm data are relatively long. Therefore, another implementation method is provided in the embodiment of the present invention, in which the pre-processing action of extracting the effective semantic words in the alarm data can be deleted, and the entire effective sentence can be directly classified. In order to reduce the impact of words irrelevant to the alarm classification on the prediction results caused by the omission of the pre-processing step. To improve the accuracy of the prediction, a possible implementation method is to add marked effective semantic words to the effective sentences in the alarm data, and the effective semantic words can be determined according to the effective semantic words determined in the effective sentences. For example, the training data of the port DOWN alarm category is: interface xxx is down. The effective sentence can be changed to: Interface xxx is down interface. Among them, xxx can represent the port number, which can reduce the frequency of words such as "is" that are irrelevant to the alarm classification. By increasing the probability of important feature elements in the training data and reducing the probability of words irrelevant to the alarm classification, the impact of words irrelevant to the alarm classification on the prediction results can be effectively reduced.
[0073] For the application scenario of the present invention, the category set is the category of the alarm. Taking the network device alarm as an example, the alarm can be divided into categories such as port down, board abnormality, OSPF abnormality, BGP abnormality, etc.
[0074] Step 203: determining the alarm category of the alarm data according to the alarm feature vector of the alarm data and the pre-trained classification model;
[0075] Among them, the classification model is determined based on the estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of historical alarm data under each alarm category is determined based on the conditional probability of each characteristic element of the historical alarm data in each alarm category.
[0076] The training samples of the classification model can be manually labeled or directly obtain historical alarm data from past production as training samples. For example, the training data can be: "interface xxx is down", and its label is the port DOWN alarm category.
[0077] The embodiment of the present invention takes the classification of naive Bayes as an example to illustrate the classification model in the embodiment of the present invention. Of course, the classification in the embodiment of the present invention can also be implemented by other classification models, which is not limited here. For example, the classification model can be a support vector machine (SVM) classification model, or a logistic regression (LogisticRegression), a decision tree, a neural network, a neighbor algorithm or a K-nearest neighbor (k-NearestNeighbor, KNN) classification model, etc., which is not limited in the embodiment of the present invention.
[0078] like Figure 3 As shown, the embodiment of the present invention provides a process of a naive Bayes classification method, including:
[0079] Step 301: Determine the item x to be classified. In combination with the above embodiment, the item x to be classified may be historical alarm data generated by the network device or newly acquired alarm data. The label of the historical alarm data may be preset or set during the training process.
[0080] Specifically, the method may further include extracting valid semantic words from the alarm data, and determining an alarm feature vector corresponding to the alarm data according to the valid semantic words of the alarm data;
[0081] Specifically, the alarm feature vector can be expressed as x={a1, a2, ..., a m}, and each alarm feature vector includes feature elements a1,…a m .
[0082] Step 302: Determine training data.
[0083] In the specific implementation process, the training data may be pre-set historical alarm data and labels of the historical alarm data; or new alarm learning data may be regenerated based on the verification results after training or running the classifier.
[0084] Step 303: Determine the alarm category set C = {y1, y2, ..., y n}, determine the estimated probability of each feature element under each category.
[0085] For the application scenario of the present invention, the category set is the alarm category. Taking the network device alarm as an example, the alarm can be divided into categories such as port down, board abnormality, OSPF abnormality, BGP abnormality, etc.
[0086] Specifically, it can be expressed as P(y1|x), P(y2|x), ..., P(y n |x).
[0087] A possible implementation method may include:
[0088] Step 1: Obtain a historical alarm data set of the network device; the historical alarm data set includes a plurality of historical alarm data and an alarm category corresponding to each historical alarm data;
[0089] Step 2: for each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data;
[0090] Step 3: for the same alarm category, determine the conditional probability of each characteristic element in the alarm category;
[0091] Step 4: for each historical alarm data, determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category according to the conditional probability of each characteristic element in the alarm feature vector of the historical alarm data in each alarm category;
[0092] Specifically, conditional probabilities may include:
[0093] P(a1|y1), P(a2|y1),..., P(a m |y1);P(a1|y2),P(a2|y2),...,P(a m |y2); ...; P(a1|y n ), P(a2|y n ), ..., P(a m |y n )
[0094] Among them, a m is the characteristic element in the x alarm characteristic vector, x={a1,a2,...,a m} is the alarm feature vector of the historical alarm data, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(a j |y i ) is the first jThe conditional probability of a feature element in the i-th alarm category;
[0095] Therefore, it can be determined that:
[0096]
[0097] Because the denominator of P(x) is i (i is an arbitrary value) is a constant, because we only need to maximize the numerator to determine P(y i Since each feature element is conditionally independent, the estimated probability of the alarm feature vector of the historical alarm data under each alarm category can be determined according to the following formula:
[0098]
[0099] Where x = {a1, a2, ..., a m} is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the i-th alarm category in the historical alarm data set; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
[0100] Here, we can assume that the probability of each fault alarm is the same, and then calculate the conditional probability of each category for each feature element. Specifically, it can be implemented according to word frequency statistics. Each valid semantic word in the alarm in each training data is counted for word frequency.
[0101] For example, the alarm category in the training data is VPN anomaly: "VPN xxx is disconnected".
[0102] Remove the word "is" which is not related to the classification. The frequency of "VPN", "xxx", and "disconnected" in VPN anomalies is 1 / 3.
[0103] For example, for all training data, the statistical value of the probability of VPN in the category of "VPN abnormality" can be determined. Similarly, the statistical value of the probability of all valid words in the category of "VPN abnormality" can be determined as the conditional probability of each feature element for each category.
[0104] It should be noted that when P(a i |yj )=0, when a certain feature element does not appear in a certain category, this phenomenon occurs, which will greatly reduce the quality of the classifier. In order to solve this problem, a possible implementation method may include:
[0105] If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
[0106] Specifically, the predetermined offset item may be 1.
[0107] Step 5: training the classification model according to the estimated probability of each historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; specifically, training all training data until the accuracy of the classifier reaches a preset threshold.
[0108] Step 304: Determine the predicted classification.
[0109] P(x|y k )P(y k ) is the largest, then the category of the item to be classified x is y k .
[0110] Specifically, if P(y k |x)=max{P(y1|x), P(y2|x),..., P(y n |x)}, then determine x∈y k .
[0111] Step 305: Verify the predicted value of the classification according to the label of the training data. If it is determined that the verification is passed, execute step 306; otherwise, generate new learning data for the training data and execute step 302;
[0112] Specifically, it may include: determining the prediction classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category;
[0113] According to the predicted classification and the alarm category corresponding to each historical alarm data, the parameters of the classification model are adjusted so that the confidence of the predicted classification is greater than a preset threshold.
[0114] Step 306: Determine the classification result of the item to be classified.
[0115] It should be noted that the above verification process can also be performed during the classification process, that is, if the training data is very limited, the classifier may not be able to correctly classify a new alarm when it first appears. Then manual correction is required. Suppose there is an alarm data of category B that has never appeared before, and it is judged as category A by the classifier, then it is necessary to manually add learning data of the alarm data, for example, usually similar alarms to this alarm, to regenerate the classifier and thus enhance the accuracy of the classifier.
[0116] Based on the same inventive concept, Figure 4 As shown, another embodiment of the present invention provides an alarm classification device, comprising:
[0117] The transceiver unit 401 is used to obtain the alarm data generated by the network device;
[0118] Processing unit 402 is used to extract valid semantic words from the alarm data, and determine the alarm feature vector corresponding to the alarm data based on the valid semantic words of the alarm data; wherein each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data; determine the alarm category of the alarm data based on the alarm feature vector of the alarm data and a pre-trained classification model; the classification model is determined based on the estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of the historical alarm data under each alarm category is determined based on the conditional probability of each feature element of the historical alarm data in each alarm category.
[0119] In a possible implementation manner, the transceiver unit 401 is further configured to:
[0120] Acquire a historical alarm data set of the network device; the historical alarm data set includes a plurality of historical alarm data and an alarm category corresponding to each historical alarm data;
[0121] The processing unit 402 is also used to: for each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data; for the same alarm category, determine the conditional probability of each feature element in the alarm category; for each historical alarm data, determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category according to the conditional probability of each feature element in the alarm feature vector of the historical alarm data in each alarm category; and train the classification model according to the estimated probability of each historical alarm data in each alarm category and the alarm category corresponding to each historical alarm data.
[0122] In a possible implementation manner, the processing unit 402 is specifically configured to:
[0123] Determine the predicted classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category; adjust the parameters of the classification model according to the predicted classification and the alarm category corresponding to each historical alarm data so that the confidence of the predicted classification is greater than a preset threshold.
[0124] In a possible implementation, the processing unit 402 is specifically configured to determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category according to the following formula:
[0125]
[0126] Where x = {a1, a2, ..., a m} is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the ith alarm category in the historical alarm data set; P(a j |y i ) is the first j The conditional probability of a feature element in the i-th alarm category; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
[0127] In a possible implementation manner, the processing unit 402 is further configured to:
[0128] If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
[0129] Based on the same inventive concept, Figure 5 As shown, a schematic diagram of the structure of an electronic device in an embodiment of the present invention.
[0130] An embodiment of the present invention provides an electronic device, wherein the electronic device may be a server or other computer device, etc. The electronic device may include a processor 710 (Center Processing Unit, CPU), a memory 720, an input device 730 and an output device 740, etc. The input device 730 may include a keyboard, a mouse, a touch screen, etc. The output device 740 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (CathodeRay Tube, CRT), etc.
[0131] The memory 720 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 710 with program instructions and data stored in the memory 720. In an embodiment of the present invention, the memory 720 may be used to store the program of the alarm classification method in the above embodiment.
[0132] The processor 710 calls the program instructions stored in the memory 720, and the processor 710 is used to execute according to the obtained program instructions:
[0133] Extract valid semantic words from the alarm data, and determine an alarm feature vector corresponding to the alarm data based on the valid semantic words of the alarm data; wherein each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data; determine the alarm category of the alarm data based on the alarm feature vector of the alarm data and a pre-trained classification model; the classification model is determined based on the estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of the historical alarm data under each alarm category is determined based on the conditional probability of each feature element of the historical alarm data in each alarm category.
[0134] In a possible implementation, the processor 710 is further used to: for each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data; for the same alarm category, determine the conditional probability of each feature element in the alarm category; for each historical alarm data, determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category according to the conditional probability of each feature element in the alarm feature vector of the historical alarm data in each alarm category; and train the classification model according to the estimated probability of each historical alarm data in each alarm category and the alarm category corresponding to each historical alarm data.
[0135] In a possible implementation manner, the processor 710 is specifically configured to:
[0136] Determine the predicted classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category; adjust the parameters of the classification model according to the predicted classification and the alarm category corresponding to each historical alarm data so that the confidence of the predicted classification is greater than a preset threshold.
[0137] In a possible implementation, the processor 710 is specifically configured to determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category according to the following formula:
[0138]
[0139] Where x = {a1, a2, ..., a m} is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the ith alarm category in the historical alarm data set; P(a j |y i ) is the first j The conditional probability of a feature element in the i-th alarm category; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
[0140] In a possible implementation manner, the processor 710 is further configured to:
[0141] If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
[0142] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0143] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0146] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0147] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. An alarm classification method, characterized in that: include: Obtain alarm data generated by network devices; Extracting valid semantic words from the alarm data, and determining an alarm feature vector corresponding to the alarm data according to the valid semantic words of the alarm data; wherein each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data; Determine the alarm category of the alarm data according to the alarm feature vector of the alarm data and a pre-trained classification model; the classification model is determined according to the estimated probability of the historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of the historical alarm data under each alarm category is determined according to the conditional probability of each characteristic element of the historical alarm data in each alarm category; When training the classification model, marked valid semantic words are added to the valid sentences in the alarm data, and the entire valid sentences are directly classified; the valid semantic words are determined based on the valid semantic words determined in the valid sentences.
2. The method according to claim 1, characterized in that The classification model is determined based on the estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data, including: Acquire a historical alarm data set of the network device; the historical alarm data set includes a plurality of historical alarm data and an alarm category corresponding to each historical alarm data; For each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data; For the same alarm category, determining the conditional probability of each characteristic element in the alarm category; For each historical alarm data, determining the estimated probability of the alarm feature vector of the historical alarm data under each alarm category according to the conditional probability of each characteristic element in the alarm feature vector of the historical alarm data under each alarm category; The classification model is trained according to the estimated probability of each historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data.
3. The method according to claim 2, characterized in that The training of the classification model according to the estimated probability of each historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data includes: Determine the prediction classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category; According to the predicted classification and the alarm category corresponding to each historical alarm data, the parameters of the classification model are adjusted so that the confidence of the predicted classification is greater than a preset threshold.
4. The method according to claim 2, characterized in that For each historical alarm data, the estimated probability of the alarm feature vector of the historical alarm data under each alarm category is determined according to the conditional probability of each characteristic element in the alarm feature vector of the historical alarm data under each alarm category, which is determined according to the following formula: Where x = {a1, a2, ..., a m } is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the ith alarm category in the historical alarm data set; P(a j |y i ) is the first j The conditional probability of a feature element in the i-th alarm category; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
5. The method according to claim 4, characterized in that The method further comprises: If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
6. An alarm classification device, characterized in that: include: A transceiver unit, used to obtain alarm data generated by network equipment; A processing unit, used to extract valid semantic words from the alarm data, and determine an alarm feature vector corresponding to the alarm data based on the valid semantic words of the alarm data; wherein each valid semantic word corresponds to a feature element in the alarm feature vector of the alarm data; determine the alarm category of the alarm data based on the alarm feature vector of the alarm data and a pre-trained classification model; the classification model is determined based on the estimated probability of historical alarm data under each alarm category and the alarm category corresponding to each historical alarm data; the estimated probability of the historical alarm data under each alarm category is determined based on the conditional probability of each feature element of the historical alarm data in each alarm category; The processing unit is also used to add marked valid semantic words to the valid sentences in the alarm data when training the classification model, and directly classify the entire valid sentences; the valid semantic words are determined based on the valid semantic words determined in the valid sentences.
7. The device according to claim 6, characterized in that The transceiver unit is further used for: Acquire a historical alarm data set of the network device; the historical alarm data set includes a plurality of historical alarm data and an alarm category corresponding to each historical alarm data; The processing unit is further used to: for each historical alarm data, extract valid semantic words in the historical alarm data, and use the valid semantic words as feature elements of the alarm feature vector of the historical alarm data; For the same alarm category, determine the conditional probability of each characteristic element in the alarm category; for each historical alarm data, determine the estimated probability of the alarm feature vector of the historical alarm data in each alarm category based on the conditional probability of each characteristic element in the alarm feature vector of the historical alarm data in each alarm category; train the classification model based on the estimated probability of each historical alarm data in each alarm category and the alarm category corresponding to each historical alarm data.
8. The device according to claim 7, characterized in that The processing unit is specifically used for: Determine the predicted classification of each historical alarm data according to the estimated probability of each historical alarm data under each alarm category; adjust the parameters of the classification model according to the predicted classification and the alarm category corresponding to each historical alarm data so that the confidence of the predicted classification is greater than a preset threshold.
9. The device according to claim 6, characterized in that The processing unit is specifically configured to determine the estimated probability of the alarm feature vector of the historical alarm data under each alarm category according to the following formula: Where x = {a1, a2, ..., a m } is the alarm feature vector of the historical alarm data, a m is the characteristic element in the x alarm feature vector, y i is the i-th alarm category; P(x|y i ) is the alarm category y i The probability of the alarm feature vector x appearing; P(y i ) is the occurrence probability of the ith alarm category in the historical alarm data set; P(a j |y i ) is the first j The conditional probability of a feature element in the i-th alarm category; The alarm category is y i The product of the conditional probabilities of each feature element appearing.
10. The device according to claim 9, characterized in that The processing unit is further used for: If it is determined that the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data under the first alarm category is zero, then the conditional probability of the first characteristic element in the alarm feature vector of the historical alarm data in the first alarm category is increased by a predetermined bias item; the first alarm category is any alarm category among the alarm categories; the first characteristic element is any characteristic element in the alarm feature vector of the historical alarm data.
11. An electronic device, characterized in that: include: at least one memory for storing program instructions; At least one processor is used to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 5 according to the obtained program instructions.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
