Method and device for alarm troubleshooting and root cause tracing in cluster based on log clustering
By performing cluster analysis and root cause analysis of alarm information in large cluster systems, combining topology structure and service dependencies, the problem of difficulty in locating faults in traditional methods is solved, and efficient and accurate troubleshooting and root cause traceability are achieved. .
Patent Information
- Application Number
- CN202411226484.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-12-06
AI Technical Summary
In large cluster systems, the magnitude of error logs increases sharply, making it difficult to quickly and effectively locate faults when alarm information is difficult. Traditional alarm troubleshooting methods are inefficient and prone to errors.
Using log clustering method, clustering analysis of alarm information from multiple dimensions, combining the topology structure and service dependencies of the cluster, root cause analysis technology is used to determine the root cause of alarm, and clustering algorithm and time series analysis are used. Realize alarm troubleshooting and root cause traceability.
It improves the efficiency and accuracy of alarm troubleshooting, can quickly and effectively locate the root cause of failure, reduce the dependence of manual analysis, and reduce the error rate.
Smart Images

Figure CN119089222A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for alarm troubleshooting and root cause tracing in a cluster based on log clustering. Background Art
[0002] In large cluster systems, due to the large number of nodes and complex services, when a system fails (such as a bug is triggered, a dependent service times out, etc.), the amount of error logs increases dramatically. In extreme cases, there may even be a phenomenon of "crazy error reporting". At this time, the contents of the error logs will bury and affect each other. When faced with error reporting, it is difficult for operation and maintenance and developers to sort out the logic for a while, and sometimes they even lose sight of one thing while focusing on another, and fail to solve the most core problem in the first place. How to quickly and effectively troubleshoot and locate the root cause of the alarm is the key to ensuring the stable operation of the system. Traditional alarm troubleshooting methods often rely on manual analysis and experience judgment, which is inefficient and prone to errors. Summary of the invention
[0003] In view of this, the present invention provides a method and device for alarm troubleshooting and root cause tracing in a cluster based on log clustering, so as to solve the problem that it is difficult to locate faults and trace the root causes from a large number of error logs in a cluster environment.
[0004] In a first aspect, the present invention provides a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering, the method comprising: Collect alarm information in the cluster; Performing cluster analysis on the alarm information from multiple dimensions; Based on the cluster analysis results, combined with the cluster topology and service dependencies, the root cause of the alarm is determined using root cause analysis technology.
[0005] The present invention provides a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering. The method classifies the alarm information in the cluster through a clustering algorithm, and combines time series analysis and root cause analysis technology to achieve alarm fault troubleshooting and root cause tracing, thereby improving the efficiency and accuracy of alarm troubleshooting.
[0006] In an optional embodiment, the method further includes: The results of the root cause analysis are presented to the user in a visual form.
[0007] In an optional implementation, the cluster analysis of the alarm information from multiple dimensions includes: Perform cluster analysis on the alarm information from the dimensions of time, machine, abnormality source, alarm log text prompt, and fault location; Extracting common features and rules of the alarm information; Group similar alarm messages into the same category.
[0008] In an optional implementation, the method of determining the root cause of the alarm using root cause analysis technology based on the cluster analysis result in combination with the cluster topology and service dependency includes: The fault tree analysis method is used to build a fault tree model according to the cluster topology and service dependency, and the root cause of the alarm is determined in combination with the cluster analysis results.
[0009] In an optional implementation, the method of determining the root cause of the alarm using root cause analysis technology based on the cluster analysis result in combination with the cluster topology and service dependency includes: Using causal graph analysis technology, a causal graph model is constructed based on the causal relationship and correlation between alarms, and the root cause of the alarm is determined through logical reasoning.
[0010] In an optional implementation, presenting the root cause analysis result to the user in a visual form includes: Generate fault tree diagrams or cause-and-effect diagrams to show the root causes and solutions to alarms.
[0011] In an optional implementation, presenting the root cause analysis result to the user in a visual form includes: Generate detailed report files or emails to notify users of relevant situations.
[0012] In a second aspect, the present invention provides a device for alarm troubleshooting and root cause tracing in a cluster based on log clustering, the device comprising: Collection module, used to collect alarm information in the cluster; A clustering module, used for performing cluster analysis on the alarm information from multiple dimensions; The troubleshooting module is used to determine the root cause of the alarm based on the cluster analysis results, combined with the cluster topology and service dependencies, using root cause analysis technology.
[0013] The present invention provides a device for alarm troubleshooting and root cause tracing in a cluster based on log clustering. The device classifies the alarm information in the cluster through a clustering algorithm, and combines time series analysis and root cause analysis technology to achieve alarm fault troubleshooting and root cause tracing, thereby improving the efficiency and accuracy of alarm troubleshooting.
[0014] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the alarm troubleshooting and root cause tracing method in a cluster based on log clustering of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the alarm troubleshooting and root cause tracing method in a cluster based on log clustering according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1 It is a flow chart of a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering according to an embodiment of the present invention; Figure 2 is a schematic diagram of a clustering method according to an environmental dimension according to an embodiment of the present invention; Figure 3 is a schematic diagram of a clustering method according to service type according to an embodiment of the present invention; Figure 4 It is a structural block diagram of an alarm troubleshooting and root cause tracing device in a cluster based on log clustering according to an embodiment of the present invention; Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0019] The embodiment of the present invention provides a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering. The alarm information in the cluster is classified by a clustering algorithm, and time series analysis and root cause analysis technology are combined to achieve alarm fault troubleshooting and root cause tracing.
[0020] According to an embodiment of the present invention, an embodiment of a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.
[0021] In this embodiment, a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering is provided. Figure 1 is a flow chart of a method for alarm troubleshooting and root cause tracing in a cluster based on log clustering according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S11, collecting alarm information in the cluster.
[0022] Specifically, the cluster management system or the log collection system collects alarm information in the cluster, including the alarm type, occurrence time, involved nodes or services, and other information.
[0023] Cluster refers to connecting multiple independent computers (called nodes or servers) together to perform tasks, process workloads, or provide services as a whole. These computer nodes communicate and collaborate with each other through the network, sharing resources and workloads to achieve higher performance, reliability, and scalability.
[0024] Step S12: performing cluster analysis on the alarm information from multiple dimensions.
[0025] Specifically, step S12 includes: Step S121 , cluster analysis is performed on the alarm information from the time dimension, machine dimension, abnormality source dimension, alarm log text prompt dimension and fault location dimension.
[0026] Step S122, extracting common features and rules of the alarm information.
[0027] Step S123: classify similar alarm information into the same category.
[0028] In the embodiment of the present invention, based on the experience of online troubleshooting, the indicators that operation and maintenance personnel usually pay attention to include time, machine (machine room, environment), abnormal source, alarm log text prompt, and fault location. Therefore, the above dimensions can be used to perform cluster analysis on the alarm information, extract the common features and rules of this type of alarm, and classify similar alarm information into the same category.
[0029] Clustering is an unsupervised learning algorithm that explores and analyzes unlabeled data to divide the samples in the data set into several non-overlapping subsets, each of which is called a "cluster". The goal of clustering is to make the samples in the same cluster as similar as possible, while the samples in different clusters are as different as possible. For clustering by environmental dimension, see Figure 2 . For clustering by service type, see Figure 3 .
[0030] Step S13, based on the cluster analysis results, combined with the cluster topology and service dependencies, the root cause of the alarm is determined using root cause analysis technology.
[0031] Specifically, step S13 can be implemented in the following two ways: Method 1: Use the fault tree analysis method to build a fault tree model based on the cluster topology and service dependencies, and determine the root cause of the alarm (i.e., the root node) based on the cluster analysis results.
[0032] Method 2: Use causal graph analysis technology to build a causal graph model based on the causal relationship and correlation between alarms, and determine the root cause of the alarm through logical reasoning.
[0033] Among them, root cause tracing is a method and technology that uses systematic analysis, reasoning and tools to find and determine the root cause of a problem or failure in a complex system. It aims to see the essence through the phenomenon and deeply explore the root cause behind the problem in order to more effectively solve the problem and prevent it from happening again.
[0034] In an optional implementation, the method for alarm troubleshooting and root cause tracing in a cluster based on log clustering further includes the following steps: Step S14: Display the result of the root cause analysis to the user in a visual form.
[0035] Specifically, step S14 can be implemented in the following two ways: Method 1: Generate a fault tree diagram or cause-and-effect diagram chart to show the root cause and solution of the alarm.
[0036] Method 2: Generate a detailed report file or send an email to notify the user of the relevant situation.
[0037] In this embodiment, a device for alarm troubleshooting and root cause tracing in a cluster based on log clustering is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0038] The present invention provides a method for checking and tracing the root cause of an alarm in a cluster based on log clustering, comprising: Collect alarm information in the cluster; perform cluster analysis on the alarm information from multiple dimensions; based on the cluster analysis results, combine the cluster topology and service dependencies, and use root cause analysis technology to determine the root cause of the alarm. Classify the alarm information in the cluster through clustering algorithms, and combine time series analysis and root cause analysis technology to achieve alarm troubleshooting and root cause tracing, improving the efficiency and accuracy of alarm troubleshooting.
[0039] This embodiment provides a device for checking alarms and tracing root causes in a cluster based on log clustering. Figure 4 As shown, including: The collection module 41 is used to collect alarm information in the cluster.
[0040] The clustering module 42 is used to perform cluster analysis on the alarm information from multiple dimensions.
[0041] The troubleshooting module 43 is used to determine the root cause of the alarm by using the root cause analysis technology based on the cluster analysis results, combined with the cluster topology and service dependencies.
[0042] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0043] The alarm troubleshooting and root cause tracing device in the cluster based on log clustering in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0044] The present invention provides a device for checking and tracing the root cause of an alarm in a cluster based on log clustering, comprising: The collection module is used to collect alarm information in the cluster. The clustering module is used to perform cluster analysis on the alarm information from multiple dimensions. The troubleshooting module is used to determine the root cause of the alarm based on the cluster analysis results, combined with the cluster topology and service dependency, using root cause analysis technology. The alarm information in the cluster is classified by clustering algorithm, and combined with time series analysis and root cause analysis technology, alarm fault troubleshooting and root cause tracing are realized, which improves the efficiency and accuracy of alarm troubleshooting.
[0045] The embodiment of the present invention also provides a computer device having the above Figure 4 The device shown is an alarm troubleshooting and root cause tracing device in a cluster based on log clustering.
[0046] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.
[0047] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0048] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.
[0049] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0050] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0051] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0052] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0053] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for alarm troubleshooting and root cause tracing in a cluster based on log clustering, characterized in that: The method comprises: Collect alarm information in the cluster; Performing cluster analysis on the alarm information from multiple dimensions; Based on the cluster analysis results, combined with the cluster topology and service dependencies, the root cause of the alarm is determined using root cause analysis technology.
2. The method for alarm investigation and root cause tracing in a cluster based on log clustering according to claim 1 is characterized in that: The method further comprises: The results of the root cause analysis are presented to the user in a visual form.
3. The method for alarm investigation and root cause tracing in a cluster based on log clustering according to claim 1 is characterized in that: The cluster analysis of the alarm information from multiple dimensions includes: Perform cluster analysis on the alarm information from the dimensions of time, machine, abnormality source, alarm log text prompt, and fault location; Extracting common features and rules of the alarm information; Group similar alarm messages into the same category.
4. The method for alarm investigation and root cause tracing in a cluster based on log clustering according to claim 1 is characterized in that: Based on the cluster analysis results, combined with the cluster topology and service dependency, the root cause of the alarm is determined using root cause analysis technology, including: The fault tree analysis method is used to build a fault tree model according to the cluster topology and service dependency, and the root cause of the alarm is determined in combination with the cluster analysis results.
5. The method for alarm investigation and root cause tracing in a cluster based on log clustering according to claim 1 is characterized in that: Based on the cluster analysis results, combined with the cluster topology and service dependency, the root cause of the alarm is determined using root cause analysis technology, including: Using causal graph analysis technology, a causal graph model is constructed based on the causal relationship and correlation between alarms, and the root cause of the alarm is determined through logical reasoning.
6. The method for alarm investigation and root cause tracing in a cluster based on log clustering according to claim 2 is characterized in that: The root cause analysis results are presented to the user in a visual form, including: Generate fault tree diagrams or cause-and-effect diagrams to show the root causes and solutions to alarms.
7. The method for alarm investigation and root cause tracing in a cluster based on log clustering according to claim 2, characterized in that: The root cause analysis results are presented to the user in a visual form, including: Generate detailed report files or emails to notify users of relevant situations.
8. A device for alarm investigation and root cause tracing in a cluster based on log clustering, characterized in that: The device comprises: Collection module, used to collect alarm information in the cluster; A clustering module, used for performing cluster analysis on the alarm information from multiple dimensions; The troubleshooting module is used to determine the root cause of the alarm based on the cluster analysis results, combined with the cluster topology and service dependencies, using root cause analysis technology.
9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the alarm troubleshooting and root cause tracing method in a cluster based on log clustering as described in any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the alarm troubleshooting and root cause tracing method in a cluster based on log clustering according to any one of claims 1 to 7.
Citation Information
Patent Citations
Intelligent fault root cause analysis method and system
CN112152830A
Method and device based on log alarm
CN113254319A
Fault root cause positioning method and device, equipment and storage medium
CN117376092A
Log data analysis method and device, computer equipment and storage medium
CN117785539A
Cited By
Unified representation and dynamic knowledge reasoning method and device for intelligent cluster system
CN120338114A
Operation and maintenance event analysis method and device based on root cause aggregation and storage medium
CN122247834A
