Abnormal data root cause analysis method and device based on text clustering combined with link call

Abstract

The present application discloses a method and device for root cause analysis of abnormal data by combining text clustering with link calls. The method is used for the above-mentioned device, including: collecting the underlying abnormal information data encapsulated by the program, and reporting the abnormal information data to the message system for consumption processing to obtain abnormal consumption data; judging whether to perform link analysis on the abnormal consumption data, if so, filtering the invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregating and classifying the filtered abnormal consumption data through a clustering algorithm to obtain abnormal cluster data; if not, aggregating and classifying the abnormal consumption data through a clustering algorithm to obtain abnormal cluster data; matching the obtained abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data. The present application combines text clustering with link calls based on the underlying abnormalities, which helps operation and maintenance personnel make maintenance decisions in a timely manner and avoid losses caused by system failures.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance analysis, and in particular to a method and device for analyzing root causes of abnormal data by combining text clustering with link calls. Background Art

[0002] The exception log is a tool for recording the running status of an application. Usually, the exception log records the on-site information when the system runs wrong and the context information of key execution points. When the business traffic is small and the logical architecture is not complex, there are fewer exception logs for failures. Operation and maintenance personnel can quickly locate the fault point and find the abnormal problem based on the error log.

[0003] However, with the increase in the complexity of business logic and the increase in dependent services, the volume of error logs has increased dramatically, and the service call links have become more complex. In some applications, the number of business exception logs can reach millions per day, and even in some extreme cases, the volume of crazy error reports triggered will be even higher. The exception logs and system alarms change rapidly and affect each other, making it difficult for operation and maintenance personnel to quickly sort out the links and find the root cause of the problem, so that the best time to restore the system is missed.

[0004] Therefore, based on the problems encountered in the existing abnormal data root cause analysis, it is necessary to provide a method to quickly help operation and maintenance personnel locate the root cause of the fault. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for analyzing root causes of abnormal data by combining text clustering with link calls, so as to solve the problem that the root causes of faults cannot be quickly located due to the large amount of data in the existing abnormal data analysis.

[0006] To achieve the above object, the present invention discloses the following technical solution:

[0007] On one hand, the present invention provides a method for analyzing root causes of abnormal data by combining text clustering with link calls, the method comprising the following steps:

[0008] Collect the bottom-level abnormal information data encapsulated by the program, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0009] Determine whether to perform link analysis on the abnormal consumption data,

[0010] If yes, firstly, invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis, and then the filtered abnormal consumption data are aggregated and classified through a clustering algorithm to obtain abnormal cluster data;

[0011] If not, clustering and classifying the abnormal consumption data through a clustering algorithm to obtain abnormal cluster data;

[0012] The obtained abnormal cluster data is matched with the set fault feature object to obtain the category of the abnormal cluster data, and the root cause of the abnormal information data is determined according to the category of the abnormal cluster data.

[0013] The above-mentioned abnormal data root cause analysis method also includes the step of establishing a fault feature library for storing the set fault feature objects.

[0014] In the above-mentioned abnormal data root cause analysis method, the step of aggregating and classifying abnormal consumption data by using a clustering algorithm further includes:

[0015] Parsing the abnormal consumption data into abnormal information text;

[0016] Preprocessing the abnormal information text, deleting invalid information in the abnormal information text, and obtaining a preprocessed text;

[0017] Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors;

[0018] The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

[0019] Furthermore, the invalid information in the exception information text includes special symbols and / or dirty data.

[0020] In the above-mentioned abnormal data root cause analysis method, the step of filtering invalid abnormal nodes in the abnormal consumption data through link analysis further comprises:

[0021] Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data;

[0022] The Nebula graph space is used to query the service link information of each abnormal node, and the invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis to obtain the problematic abnormal nodes.

[0023] Furthermore, the fault feature object is generated based on keywords in the abnormal information data and the abnormal information keywords reported by the system.

[0024] The above abnormal data root cause analysis method further includes the following steps before matching the abnormal clustering data with the set fault feature object:

[0025] Setting up a custom word library and adding custom keywords to the custom word library;

[0026] A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

[0027] The above-mentioned abnormal data root cause analysis method, the step of matching the abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data further includes:

[0028] Get the local model file and create a classification model;

[0029] The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

[0030] Another aspect of the present invention provides a device for analyzing root causes of abnormal data by combining text clustering with link calls, the device comprising:

[0031] A collection module is used to collect the bottom-level abnormal information data encapsulated by the program, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0032] A judgment module, used to judge whether to perform link analysis on the abnormal consumption data;

[0033] An analysis and clustering module, which is used to, when it is determined that a link analysis is to be performed on the abnormal consumption data, first filter invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregate and classify the filtered abnormal consumption data through a clustering algorithm to obtain abnormal clustering data;

[0034] A clustering module, for clustering and classifying the abnormal consumption data by a clustering algorithm to obtain abnormal cluster data when it is determined that no link analysis is to be performed on the abnormal consumption data;

[0035] The matching module is used to match the obtained abnormal clustering data with the set fault feature object, obtain the category of the abnormal clustering data, and perform root cause analysis of the abnormal information data.

[0036] The above-mentioned abnormal data root cause analysis device further includes:

[0037] The feature library module is used to establish a fault feature library to store the set fault feature objects.

[0038] The above-mentioned abnormal data root cause analysis device, the analysis clustering module or the clustering module aggregates and classifies the abnormal consumption data through a clustering algorithm, including:

[0039] Parsing the abnormal consumption data into abnormal information text;

[0040] Preprocessing the abnormal information text, deleting invalid information in the abnormal information text, and obtaining a preprocessed text;

[0041] Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors;

[0042] The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

[0043] In the above-mentioned abnormal data root cause analysis device, the analysis and clustering module filters invalid abnormal nodes in the abnormal consumption data through link analysis, including:

[0044] Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data;

[0045] The service link information of each abnormal node is queried using Nebula graph space, and invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis to obtain problematic abnormal nodes.

[0046] In the above-mentioned abnormal data root cause analysis device, the matching module is also used to obtain a fault feature object from a fault feature library, including:

[0047] Setting up a custom word library and adding custom keywords to the custom word library;

[0048] A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

[0049] In the above-mentioned abnormal data root cause analysis device, the matching module matches the abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data, including:

[0050] Get the local model file and create a classification model;

[0051] The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

[0052] In another aspect, the present invention provides a system for network intelligent operation and maintenance analysis, the system comprising the abnormal data root cause analysis device combining text clustering with link call as described above.

[0053] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the abnormal data root cause analysis method combining text clustering with link calling as described above is implemented.

[0054] In addition, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for root cause analysis of abnormal data combining text clustering with link calls when executing the computer program.

[0055] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0056] In the embodiment of the present application, the bottom-level abnormal information data encapsulated by the acquisition program is combined with text clustering and link calling. When it is determined that link analysis is required, invalid abnormal nodes in abnormal consumption data are first filtered through link analysis, and then the filtered abnormal consumption data is aggregated and classified through a clustering algorithm. When it is determined that link analysis is not required, the abnormal consumption data is directly aggregated and classified through a clustering algorithm, so as to achieve a rapid analysis of the problems occurring in the system based on the bottom-level abnormalities and from a global level. The abnormal clustering data obtained by aggregation and classification is matched with the set fault feature objects, so that the root cause of the system failure can be quickly determined, which helps operation and maintenance personnel make maintenance decisions in a timely manner and avoid losses caused by system failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0058] Figure 1 A schematic diagram of a process flow of a method for analyzing root causes of abnormal data by combining text clustering with link calls provided in an embodiment of the present application;

[0059] Figure 2 A schematic flow chart of a method for analyzing root causes of abnormal data by combining text clustering with link calls provided in another embodiment of the present application;

[0060] Figure 3 for Figure 2 Schematic diagram of the specific implementation of step S270;

[0061] Figure 4 A schematic diagram of the structure and principle of an abnormal data root cause analysis device combining text clustering with link call provided in one embodiment of the present application;

[0062] Figure 5 A schematic diagram of the system structure principle for network intelligent operation and maintenance analysis provided in one embodiment of the present application;

[0063] Figure 6A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] It should be noted that references to "one embodiment", "embodiment", "example embodiment", etc. in this specification refer to the embodiment described, which may include specific features, structures or characteristics, but not every embodiment must include these specific features, structures or characteristics. In addition, such expressions do not refer to the same embodiment. Furthermore, when describing specific features, structures or characteristics in conjunction with an embodiment, whether or not there is an explicit description, it has been shown that it is within the knowledge of those skilled in the art to combine such features, structures or characteristics into other embodiments.

[0066] In addition, certain words are used in the specification and subsequent claims to refer to specific components or parts. Those with ordinary knowledge in the relevant field should understand that manufacturers can use different nouns or terms to refer to the same component or part. This specification and subsequent claims do not use differences in names as a way to distinguish components or parts, but use differences in the functions of components or parts as the criteria for distinction. "Including" and "including" mentioned throughout the specification and subsequent claims are open-ended terms and should be interpreted as "including but not limited to". In addition, the word "connected" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connection through other devices.

[0067] refer to Figure 1 , Figure 1 A schematic flow chart of a method for analyzing root causes of abnormal data by combining text clustering with link calls provided in an embodiment of the present application is shown, and the method comprises the following steps:

[0068] S110, collecting the bottom-level abnormal information data encapsulated by the program, and reporting the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0069] S120, determining whether to perform link analysis on the abnormal consumption data;

[0070] S130: If yes, firstly filter invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregate and classify the filtered abnormal consumption data through a clustering algorithm to obtain abnormal cluster data;

[0071] S140, if not, clustering and classifying the abnormal consumption data by using a clustering algorithm to obtain abnormal cluster data;

[0072] S150 , matching the obtained abnormal cluster data with a set fault feature object to obtain a category of the abnormal cluster data, and determining a root cause of the abnormal information data according to the category of the abnormal cluster data.

[0073] As mentioned above, due to the increase in the complexity of existing business logic, the service call links are complicated, and the number of error logs has increased dramatically, making it difficult for operation and maintenance personnel to quickly find the root cause of the fault. In this embodiment, based on the underlying abnormality, text clustering is combined with link calls, abnormal consumption data is aggregated and classified through a clustering algorithm, and the abnormal cluster data is matched with the set fault feature object to obtain the category of the abnormal cluster data. The operation and maintenance personnel can comprehensively and quickly determine the root cause of the system fault according to the category of the abnormal cluster data, thereby helping the operation and maintenance personnel to make maintenance decisions in a timely manner and reduce system failure losses.

[0074] refer to Figure 2 , Figure 2 A schematic flow chart of a method for analyzing root causes of abnormal data by combining text clustering with link calls provided in another embodiment of the present application is shown, and the method comprises the following steps:

[0075] S210, collecting the bottom-level abnormal information data encapsulated by the program, and reporting the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0076] In a specific implementation, the message system may be a Kafka distributed publish-subscribe message system, which reports abnormal information data to the Topic of Kafka through a monitoring engine program.

[0077] S220, determining whether to perform link analysis on the abnormal consumption data;

[0078] S230: If yes, firstly filter invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregate and classify the filtered abnormal consumption data through a clustering algorithm to obtain abnormal cluster data;

[0079] In actual operation, many errors in the abnormal information data are related to the abnormalities between the service link calls. Usually in the event of an accident, a large number of services generate fault warning messages, and it is difficult for the operation and maintenance personnel to find the root cause. Therefore, it is necessary to perform a full-link analysis of the warning dimension based on the service link call relationship to find the source of the faulty service. In the specific implementation, the real-time performance of Flink and the Nebula graph database can be used to realize the construction of the service link. In some embodiments, filtering the invalid abnormal nodes in the abnormal consumption data through link analysis can specifically include:

[0080] Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data;

[0081] The Nebula graph space is used to query the service link information of each abnormal node, and the invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis to obtain the problematic abnormal nodes.

[0082] After link analysis, invalid abnormal nodes in abnormal consumption data are filtered out, problem abnormal nodes are found, and then abnormal data is re-clustered based on the problem abnormal nodes. Link analysis can greatly reduce invalid abnormal information, helping operation and maintenance personnel to quickly locate problem nodes and solve abnormalities in a timely manner.

[0083] S240, if not, clustering and classifying the abnormal consumption data by using a clustering algorithm to obtain abnormal cluster data;

[0084] Abnormal consumption data usually contains abnormal text, source service information, cluster information, etc. Generally, you need to read all the data to obtain valuable information, which is often some keywords. The core of this operation step is to segment the abnormal text and then aggregate and classify the keywords through clustering operations.

[0085] In some implementations, the clustering and classification of abnormal consumption data by a clustering algorithm in step S230 and step S240 may specifically include:

[0086] Parsing the abnormal consumption data into abnormal information text;

[0087] Preprocessing the exception information text, deleting invalid information in the exception information text, including special symbols, dirty data, etc., to obtain a preprocessed text;

[0088] Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors;

[0089] The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

[0090] S250, establishing a fault feature library for storing the set fault feature objects;

[0091] In a specific implementation, the fault feature object is generated based on the keywords in the abnormal information data and the abnormal information keywords reported by the system.

[0092] S260, acquiring a fault feature object from a fault feature library, matching the obtained abnormal clustering data with the fault feature object, obtaining a category of the abnormal clustering data, and determining a root cause of the abnormal information data according to the category of the abnormal clustering data, and generating aggregated data;

[0093] In some implementations, obtaining a fault feature object from a fault feature library may specifically include:

[0094] Setting up a custom word library and adding custom keywords to the custom word library;

[0095] A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

[0096] In a specific implementation, the abnormal clustering data is matched with the fault feature object to obtain the category of the abnormal clustering data, which may specifically include:

[0097] Get the local model file and create a classification model;

[0098] Matching the abnormal cluster data with the assigned fault feature object, and scoring the matching similarity using the classification model, and determining the category of the abnormal cluster data according to a group of matches with a high score;

[0099] S270, displaying the aggregated data through an interface;

[0100] Figure 3 for Figure 2 The specific implementation diagram of step S270 is as follows: Figure 3 As shown, in some implementations, the aggregated data is displayed through an interface in step S270, which may specifically include:

[0101] S310, data cache: write the aggregated data into Kafka through the Kafka producer;

[0102] S320, Data storage: Use Flume program to consume historical data and relational data in Kafka, write to ES, and display abnormal situations through the details interface;

[0103] S330, Data display: Write cluster data into InfluxDB, use Grafana to display the data stored in InfluxDB, observe the trend changes of cluster data, and drill into the details interface through the Panel links in Grafana to query historical related data.

[0104] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0105] refer to Figure 4 , Figure 4 The schematic diagram of the structure principle of the abnormal data root cause analysis device for text clustering combined with link call provided by an embodiment of the present application is shown. The root cause analysis of abnormal data can be realized by using the device, and the device described below can be referred to in correspondence with the method described above. The device 400 includes:

[0106] The collection module 410 is used to collect the bottom-level abnormal information data of the program encapsulation, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0107] A judgment module 420 is used to judge whether to perform link analysis on the abnormal consumption data;

[0108] The analysis and clustering module 430 is used for, when it is determined that the abnormal consumption data needs to be subjected to link analysis, first filtering invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregating and classifying the filtered abnormal consumption data through a clustering algorithm to obtain abnormal clustering data;

[0109] A clustering module 440 is used to aggregate and classify the abnormal consumption data by a clustering algorithm to obtain abnormal cluster data when it is determined that the abnormal consumption data does not need to be subjected to link analysis;

[0110] The matching module 450 is used to match the obtained abnormal cluster data with the set fault feature object, obtain the category of the abnormal cluster data, and perform root cause analysis of the abnormal information data.

[0111] In some embodiments, the device may further include:

[0112] The feature library module is used to establish a fault feature library to store the set fault feature objects, and the fault feature objects are generated based on keywords in the abnormal information data and the abnormal information keywords reported by the system.

[0113] In some implementations, the analysis and clustering module 430 or the clustering module 440 aggregates and classifies the abnormal consumption data using a clustering algorithm, which may specifically include:

[0114] Parsing the abnormal consumption data into abnormal information text;

[0115] Preprocessing the exception information text, deleting invalid information in the exception information text, including special symbols, dirty data, etc., to obtain a preprocessed text;

[0116] Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors;

[0117] The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

[0118] In some implementations, the analysis and clustering module 430 filters invalid abnormal nodes in the abnormal consumption data through link analysis, which may specifically include:

[0119] Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data;

[0120] The service link information of each abnormal node is queried using Nebula graph space, and invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis to obtain problematic abnormal nodes.

[0121] In a specific implementation, the matching module 450 is further used to obtain a fault feature object from a fault feature library, including:

[0122] Setting up a custom word library and adding custom keywords to the custom word library;

[0123] A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

[0124] In some implementations, the matching module 450 matches the abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data, which may specifically include:

[0125] Get the local model file and create a classification model;

[0126] The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

[0127] Based on the abnormal data root cause analysis device combining text clustering with link call in the above embodiment, the present application also provides a system for network intelligent operation and maintenance analysis. Figure 5 As shown, Figure 5 A schematic diagram of the system structure principle for network intelligent operation and maintenance analysis provided in an embodiment of the present application, wherein the system 500 for network intelligent operation and maintenance analysis includes a client 510 and a server 520, wherein the client 510 may be a user terminal such as a smart phone or a computer, and is used to display the root cause analysis of abnormal data and the change trend. The client 510 is connected to the server 520 for communication via a network, and the abnormal data root cause analysis device 400 in the above embodiment is disposed in the server 510.

[0128] The present application also provides a computer-readable storage medium for storing the execution of Figures 1 to 3 A computer program of any of the aforementioned text clustering combined with link call abnormal data root cause analysis methods. For example, a computer program instruction, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. The program instruction for calling the method of the present application may be stored in a fixed or removable storage medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium and / or stored in a storage medium that runs according to the program instruction.

[0129] In addition, the present application also provides an electronic device, in some embodiments, such as Figure 6 As shown, Figure 6 The electronic device 600 provided in one embodiment of the present application is a schematic diagram of the structure of the electronic device 600, which may include an input unit 610, a memory 620, a processor 630 and an output unit 640. The memory 620 stores program instructions that can be run on the processor 630, and the processor 630 calls the program instructions to execute the methods and / or technical solutions based on the aforementioned multiple embodiments. The electronic device 600 may be a mobile terminal device such as a mobile phone or a computer.

[0130] In summary, the text clustering combined with the abnormal data root cause analysis method, system, storage medium and electronic device provided by the present application, after the running framework of each client machine captures the exception, it will recursively obtain the underlying exception, so that the exception close to the root cause can be directly obtained on the client, laying the foundation for rapid analysis for finding the root cause of the exception, and then reporting these underlying exceptions to the server for analysis and processing. When the server performs analysis and processing, combined with the service link call relationship, the source caller on the service call link is further filtered out, and then the remaining abnormal information is processed. After clustering, the real abnormal root cause is obtained, and a rapid analysis of the problems occurring in the system is achieved from a global level. When performing rapid abnormal root cause analysis, by matching with the fault feature object in the fault feature library, the root cause of the system fault can be quickly determined, which helps operation and maintenance personnel to make maintenance decisions in a timely manner and stop losses in a timely manner.

[0131] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.

[0132] Although the present application has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements may be made to the present application. Therefore, these modifications or improvements made without departing from the spirit of the present application all fall within the scope of protection claimed in the present application.

[0133] The present application discloses A1, a method for analyzing root causes of abnormal data by combining text clustering with link calling, comprising the following steps:

[0134] Collect the bottom-level abnormal information data encapsulated by the program, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0135] Determine whether to perform link analysis on the abnormal consumption data,

[0136] If yes, firstly, invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis, and then the filtered abnormal consumption data are aggregated and classified through a clustering algorithm to obtain abnormal cluster data;

[0137] If not, clustering and classifying the abnormal consumption data through a clustering algorithm to obtain abnormal cluster data;

[0138] The obtained abnormal cluster data is matched with the set fault feature object to obtain the category of the abnormal cluster data, and the root cause of the abnormal information data is determined according to the category of the abnormal cluster data.

[0139] A2. The method according to A1 further includes the step of establishing a fault feature library for storing the set fault feature objects.

[0140] A3. According to the method described in A1, the step of aggregating and classifying abnormal consumption data by using a clustering algorithm further comprises:

[0141] Parsing the abnormal consumption data into abnormal information text;

[0142] Preprocessing the abnormal information text, deleting invalid information in the abnormal information text, and obtaining a preprocessed text;

[0143] Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors;

[0144] The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

[0145] A4. According to the method described in A3, the invalid information in the abnormal information text includes special symbols and / or dirty data.

[0146] A5. According to the method described in A1, the step of filtering invalid abnormal nodes in the abnormal consumption data through link analysis further comprises:

[0147] Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data;

[0148] The Nebula graph space is used to query the service link information of each abnormal node, and the invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis to obtain the problematic abnormal nodes.

[0149] A6. According to the method described in A1, the fault feature object is generated based on keywords in the abnormal information data and the abnormal information keywords reported by the system.

[0150] A7. The method according to A2 further includes the following steps before matching the abnormal clustering data with the set fault feature object:

[0151] Setting up a custom word library and adding custom keywords to the custom word library;

[0152] A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

[0153] A8. According to the method described in A7, the step of matching the abnormal clustering data with the set fault feature object to obtain the category of the abnormal clustering data further includes:

[0154] Get the local model file and create a classification model;

[0155] The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

[0156] The present application also discloses B9, a device for analyzing root causes of abnormal data by combining text clustering with link calls, the device comprising:

[0157] A collection module is used to collect the bottom-level abnormal information data encapsulated by the program, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data;

[0158] A judgment module, used to judge whether to perform link analysis on the abnormal consumption data;

[0159] An analysis and clustering module, which is used to, when it is determined that a link analysis is to be performed on the abnormal consumption data, first filter invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregate and classify the filtered abnormal consumption data through a clustering algorithm to obtain abnormal clustering data;

[0160] A clustering module, for clustering and classifying the abnormal consumption data by a clustering algorithm to obtain abnormal cluster data when it is determined that no link analysis is to be performed on the abnormal consumption data;

[0161] The matching module is used to match the obtained abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data, and determine the root cause of the abnormal information data according to the category of the abnormal cluster data.

[0162] B10. The device according to B9, further comprising:

[0163] The feature library module is used to establish a fault feature library to store the set fault feature objects.

[0164] B11. According to the device described in B9, the analysis and clustering module or the clustering module aggregates and classifies the abnormal consumption data through a clustering algorithm, including:

[0165] Parsing the abnormal consumption data into abnormal information text;

[0166] Preprocessing the abnormal information text, deleting invalid information in the abnormal information text, and obtaining a preprocessed text;

[0167] Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors;

[0168] The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

[0169] B12. According to the device described in B9, the analysis and clustering module filters invalid abnormal nodes in the abnormal consumption data through link analysis, including:

[0170] Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data;

[0171] The Nebula graph space is used to query the service link information of each abnormal node, and the invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis to obtain the problematic abnormal nodes.

[0172] B13. According to the device described in B10, the matching module is further used to obtain a fault feature object from the fault feature library, including:

[0173] Setting up a custom word library and adding custom keywords to the custom word library;

[0174] A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

[0175] B14. According to the device described in B13, the matching module matches the abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data, including:

[0176] Get the local model file and create a classification model;

[0177] The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

[0178] The present application also discloses C15, a system for network intelligent operation and maintenance analysis, the system comprising the abnormal data root cause analysis device of text clustering combined with link call described in any one of B9 to B14.

[0179] The present application also discloses D16, a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the abnormal data root cause analysis method of text clustering combined with link calling described in any one of A1 to A8 is implemented.

[0180] The present application also discloses E17, an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the abnormal data root cause analysis method combining text clustering with link calls as described in any one of A1 to A8 is implemented.

Claims

1. A method for analyzing abnormal data root causes by combining text clustering with link calls, characterized in that: The following steps are involved: Collect the bottom-level abnormal information data encapsulated by the program, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data; Determine whether to perform link analysis on the abnormal consumption data, If yes, firstly, invalid abnormal nodes in the abnormal consumption data are filtered out through link analysis, and then the filtered abnormal consumption data are aggregated and classified through a clustering algorithm to obtain abnormal cluster data; If not, clustering and classifying the abnormal consumption data through a clustering algorithm to obtain abnormal cluster data; Matching the obtained abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data, and determining the root cause of the abnormal information data according to the category of the abnormal cluster data; Wherein, filtering invalid abnormal nodes in the abnormal consumption data through link analysis includes: Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data; Use Nebula graph space to query the service link information of each abnormal node, filter invalid abnormal nodes in the abnormal consumption data through link analysis, and obtain the problematic abnormal nodes; The step of matching the abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data includes: Get the local model file and create a classification model; The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

2. The abnormal data root cause analysis method combining text clustering and link call according to claim 1 is characterized in that: Also includes: The step of establishing a fault feature library is used to store the set fault feature objects.

3. The abnormal data root cause analysis method combining text clustering and link call according to claim 1 is characterized in that: The step of aggregating and classifying abnormal consumption data by using a clustering algorithm further includes: Parsing the abnormal consumption data into abnormal information text; Preprocessing the abnormal information text, deleting invalid information in the abnormal information text, and obtaining a preprocessed text; Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors; The word vectors are aggregated and classified by a clustering algorithm to obtain abnormal clustering data.

4. The abnormal data root cause analysis method combining text clustering and link calling according to claim 2 is characterized in that: Before matching the abnormal clustering data with the set fault feature object, the following steps are also included: Setting up a custom word library and adding custom keywords to the custom word library; A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

5. A device for analyzing root causes of abnormal data by combining text clustering with link calls, characterized in that: The device comprises: A collection module is used to collect the bottom-level abnormal information data encapsulated by the program, and report the abnormal information data to the message system for consumption processing to obtain abnormal consumption data; A judgment module, used to judge whether to perform link analysis on the abnormal consumption data; An analysis and clustering module, which is used to, when it is determined that a link analysis is to be performed on the abnormal consumption data, first filter invalid abnormal nodes in the abnormal consumption data through link analysis, and then aggregate and classify the filtered abnormal consumption data through a clustering algorithm to obtain abnormal clustering data; A clustering module, for clustering and classifying the abnormal consumption data by a clustering algorithm to obtain abnormal cluster data when it is determined that no link analysis is to be performed on the abnormal consumption data; a matching module, used to match the obtained abnormal cluster data with a set fault feature object to obtain the category of the abnormal cluster data, and determine the root cause of the abnormal information data according to the category of the abnormal cluster data; and / or, A feature library module, used to establish a fault feature library to store the set fault feature objects; The analysis and clustering module filters invalid abnormal nodes in the abnormal consumption data through link analysis, including: Processing the abnormal consumption data to obtain all abnormal nodes in the abnormal consumption data; Use Nebula graph space to query the service link information of each abnormal node, filter invalid abnormal nodes in the abnormal consumption data through link analysis, and obtain the problematic abnormal nodes; The matching module matches the abnormal cluster data with the set fault feature object to obtain the category of the abnormal cluster data, including: Get the local model file and create a classification model; The abnormal cluster data is matched with the fault feature object after the assignment, and the matching similarity is scored using the classification model, and the category of the abnormal cluster data is determined according to a group of matches with high scores.

6. The abnormal data root cause analysis device combining text clustering with link call according to claim 5, characterized in that: The analysis clustering module or the clustering module aggregates and classifies the abnormal consumption data through a clustering algorithm, including: Parsing the abnormal consumption data into abnormal information text; Preprocessing the abnormal information text, deleting invalid information in the abnormal information text, and obtaining a preprocessed text; Segmenting the preprocessed text, and extracting feature words after segmentation to generate word vectors; Aggregate and classify the word vectors using a clustering algorithm to obtain abnormal clustering data; or, The matching module is also used to obtain a fault feature object from the fault feature library, including: Setting up a custom word library and adding custom keywords to the custom word library; A fault feature object is obtained from the fault feature library, and a value is assigned to the fault feature object based on the user-defined keyword.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the abnormal data root cause analysis method combining text clustering with link calling according to any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the abnormal data root cause analysis method combining text clustering with link calling according to any one of claims 1 to 5 is implemented.

Images