Root cause analysis method and device

Abstract

The invention discloses a root cause analysis method and device, and relates to the technical field of big data. A specific embodiment of the method comprises the steps of obtaining a running log and resource data of a task; extracting an exception list of the task from the running log by utilizing a pre-configured rule engine; and based on the exception list and the resource data, determining a root cause analysis result of the task by utilizing a pre-constructed task fingerprint database. According to the embodiment of the invention, the time cost of manual processing can be reduced, the processing efficiency of root cause analysis is improved, and the accuracy of a root cause analysis result is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a root cause analysis method and device. Background Art

[0002] In the data processing process, there are usually more than one abnormal error in the task, and it is generally necessary to analyze the root cause of the abnormality to repair the task. At present, the root cause of the error is mainly analyzed through artificial intelligence and experience, or the root cause analysis results of the task are obtained by using a trained large model.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the related art:

[0004] The current root cause analysis method requires a lot of manpower and time costs, has low efficiency, and the accuracy of the root cause analysis results is poor. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a root cause analysis method and device, which can reduce the time cost of manual processing, improve the processing efficiency of root cause analysis, and enhance the accuracy of root cause analysis results.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a root cause analysis method is provided, which is characterized by comprising:

[0007] Get the task's running log and resource data;

[0008] Use the pre-configured rule engine to extract the task exception list from the run log;

[0009] Based on the anomaly list and resource data, the root cause analysis results of the task are determined using the pre-built task fingerprint library.

[0010] Optionally, based on the exception list and resource data, a pre-built task fingerprint library is used to determine the root cause analysis results of the task, including:

[0011] The exception list and resource data are standardized respectively to generate an exception feature text and a data feature text; wherein the standardization process includes formatting process and normalization process;

[0012] The abnormal feature text and the data feature text are spliced ​​together, and the fingerprint string of the task is determined according to the spliced ​​feature text;

[0013] The root cause analysis result of the task is determined based on the fingerprint string of the task and a pre-built task fingerprint library.

[0014] Optionally, a root cause analysis result of the task is determined based on the fingerprint string of the task and a pre-built task fingerprint library, including:

[0015] Traverse the fingerprint strings in the task fingerprint library to determine the fingerprint strings similar to the fingerprint string of the task;

[0016] In response to determining a similar fingerprint string to the fingerprint string of the task, determining a root cause analysis result corresponding to the similar fingerprint string as the root cause analysis result of the task;

[0017] In response to a fingerprint string similar to the fingerprint string of the task that has not been determined, the exception class corresponding to the exception list and the weight of each exception class are determined based on a pre-built exception rule library, and the root cause analysis result of the task is determined according to the exception class corresponding to the exception list and the weight of each exception class.

[0018] Optionally, determining a fingerprint string similar to the fingerprint string of the task includes:

[0019] Calculate the Hamming distance between the fingerprint string of the task and the fingerprint string in the task fingerprint library to obtain the Hamming distance value;

[0020] In response to the Hamming distance value being less than a preset Hamming distance threshold, the fingerprint string in the task fingerprint library is determined as a similar fingerprint string to the fingerprint string of the task.

[0021] Optionally, after determining the root cause analysis result of the task according to the exception class corresponding to the exception list and the weight of each exception class, the following is further included:

[0022] Send the root cause analysis results of the task to the user and receive feedback from the user;

[0023] In response to the feedback result being satisfactory, storing the fingerprint string and the root cause analysis result of the task in the fingerprint library;

[0024] In response to the feedback result being unsatisfactory, the following steps are repeated until the number of unsatisfactory feedbacks reaches a preset feedback threshold or the feedback result is satisfactory again: the weight of the exception class is adjusted using a pre-configured optimization strategy, the root cause analysis result of the task is redetermined according to the adjusted weight of the exception class, the root cause analysis result of the redetermined task is sent to the user again, and the user's feedback result is received again.

[0025] Optionally, the method further comprises:

[0026] In response to the number of unsatisfactory feedbacks reaching a preset feedback threshold, the operation log and resource data of the task are sent to the operation and maintenance personnel, and the root cause analysis result of the task by the operation and maintenance personnel is received, and the root cause analysis result of the task by the operation and maintenance personnel is determined as the root cause analysis result of the task.

[0027] According to a second aspect of an embodiment of the present invention, there is provided a root cause analysis device, comprising:

[0028] The acquisition module is used to obtain the task's running log and resource data;

[0029] An extraction module, used to extract a task exception list from the running log using a pre-configured rule engine;

[0030] The determination module is used to determine the root cause analysis result of the task based on the exception list and resource data using a pre-built task fingerprint library.

[0031] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0032] one or more processors;

[0033] a memory for storing one or more programs,

[0034] When one or more programs are executed by one or more processors, the one or more processors are enabled to implement the method of any of the above embodiments.

[0035] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method of any of the above embodiments is implemented.

[0036] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, including a computer program, which implements the method of any of the above embodiments when the computer program is executed by a processor.

[0037] An embodiment of the above invention has the following advantages or beneficial effects: by obtaining the operation log and resource data of the task; using a pre-configured rule engine to extract the exception list of the task from the operation log; based on the exception list and resource data, using a pre-built task fingerprint library to determine the root cause analysis result of the task; thereby, the time cost of manual processing can be reduced, the processing efficiency of the root cause analysis can be improved, and the accuracy of the root cause analysis result can be improved.

[0038] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0040] Figure 1 is a schematic diagram of the main process of the root cause analysis method according to an embodiment of the present invention;

[0041] Figure 2 It is a schematic diagram of the main links of the root cause analysis of big data abnormal tasks according to a preferred embodiment of the present invention;

[0042] Figure 3 It is a schematic diagram of the main process of root cause analysis of big data abnormal tasks according to a preferred embodiment of the present invention;

[0043] Figure 4 is a schematic diagram of main modules of a root cause analysis device according to an embodiment of the present invention;

[0044] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0045] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0047] It should be noted that the acquisition, storage and application of personal information etc. involved in the embodiments of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0048] During data processing, there are usually more than one abnormal error in a task, and it is generally necessary to analyze the root cause of the abnormality to repair the task. At present, the root cause of the error is mainly analyzed through human wisdom and experience, or the root cause analysis results of the task are obtained by using a well-trained large model, which requires a lot of manpower and time costs, is inefficient, and the accuracy of the root cause analysis results is poor.

[0049] In view of this, according to one aspect of an embodiment of the present invention, a root cause analysis method is provided.

[0050] Figure 1 FIG. 1 is a schematic diagram of the main process of the root cause analysis method according to an embodiment of the present invention. Figure 1 As shown, the root cause analysis method according to the embodiment of the present invention includes the following steps S101 to S103.

[0051] Step S101, obtaining the operation log and resource data of the task.

[0052] In the big data processing environment, a task refers to a job or application submitted by a user to a computing cluster for execution. The task processes large-scale data through a computing framework. A task can be a one-time job or a batch process that is executed periodically. During the execution of a task, the system will generate a large amount of running logs and resource data. The running log is the detailed information recorded during the execution of the task. The running log includes the execution time of the task, the execution progress of each stage, the status of node communication, and the error or warning information generated during the execution of the task. For example, when a task encounters problems such as insufficient memory or calculation failure during execution, these information can be recorded in the running log. Specifically, the running log of the task can be automatically obtained by subscribing to the log service of YARN (Yet Another Resource Negotiator). By integrating with the log service of YARN, the running log of the task can be collected in real time or batches. In addition, the running log of the historical task can be read from the JobHistory (Job History Server) service. JobHistory stores the entire process information of each task from submission to completion, and stores the specific execution details of the task.

[0053] Resource data records the actual use of system resources by tasks during operation, including consumption of resources such as memory, CPU (central processing unit), network traffic, and disk I / O (input / output). Resource data can be used to understand the performance of tasks and the rationality of resource allocation. For example, the allocation and usage of memory, CPU utilization, frequency and time of garbage collection, data processing volume and running time of tasks during task operation are all key resource indicators. These data can not only reflect whether the tasks have fully utilized the allocated resources, but also reveal whether there are bottlenecks in the use of resources. Obtaining these resource data can be achieved through the cluster monitoring system, integrated with monitoring tools, and using monitoring tools to monitor and record the resource usage of tasks on each node in real time. In addition, resource data can also be obtained by calling the API (Application Programming Interface) of the relevant system.

[0054] Step S102: extracting a task exception list from the running log using a pre-configured rule engine.

[0055] A rule engine is a system based on a set of predefined rules. During the task operation, the rule engine can extract key information, such as exceptions, from the operation log according to the set rules. A preconfigured rule engine can include a rule base and the engine itself. The rule base contains rule definitions and classifications for various exception items, such as specific matching patterns for exception items such as insufficient memory and execution timeout. The rule engine can automatically analyze the operation log according to these rules and determine the task exception based on the matched rules. In the process of preconfiguring the rule engine, a series of common exception rules are first defined. These exception rules can be configured based on the experience data and analysis results of past tasks. For example, for common memory overflow exceptions, the rule base can define a matching pattern, such as the exception information of "java.lang.OutOfMemoryError"; the configuration process also includes the exception class and weight to which each exception item belongs.

[0056] The rule engine can parse the task's running log, analyze the error code, exception stack information, etc. in the running log, and match which rules are applicable to the current task log. For example, "java.lang.OutOfMemoryError" that may appear in the running log will trigger the rule engine's memory overflow rule, thereby determining that the task has a memory overflow exception. The exception list is a collection of all exception items extracted from the log by the rule engine. The exception list can also include the exception class triggered during the task running and its weight. The weight indicates the degree of influence of each exception class on the overall task error.

[0057] Step S103, based on the exception list and resource data, a root cause analysis result of the task is determined using a pre-built task fingerprint library.

[0058] The task fingerprint library is used to store the fingerprint information of the task and the corresponding root cause analysis results. The task fingerprint is a way to uniquely represent the abnormal characteristics and resource characteristics of the task, which can be generated by a hash algorithm. The task fingerprint library can be built based on historical task data. By analyzing the logs of a large number of tasks in the past, fingerprint information is generated, and these fingerprints are associated with the corresponding root causes and stored. In the actual task running process, if a new task has an abnormality, the system can quickly identify similar tasks by comparing the fingerprint of the new task with the existing fingerprints in the fingerprint library, and provide possible solutions based on the historical root causes. For example, the running logs and resource data of historical tasks are collected, and the abnormal list and resource characteristics of the task are extracted. These characteristics are converted into a unique fingerprint string through algorithms such as SimHash, usually a 64-bit or 128-bit hash value. These fingerprints are associated with the abnormal root causes of the tasks and stored in the fingerprint library, forming a mapping from task fingerprints to root cause analysis results.

[0059] Specifically, based on the correlation between abnormal events and resource data in the abnormal list, the time, type and affected resources of each abnormal event in the abnormal list are first extracted; the historical data of these resources are analyzed to find out the resource load when the abnormal event occurs, and to detect whether there is a sudden resource consumption or bottleneck. Through the predefined patterns in the task fingerprint library, the resource consumption curve of the current task is compared with the historical abnormal patterns in the fingerprint library to determine the task fingerprint that best matches the abnormal event, thereby deriving possible abnormal root causes, such as insufficient resources, thread deadlock or hardware failure; through comparison, the system can output the root cause analysis results, indicating which resource or abnormal type caused the failure or abnormal behavior of the current task.

[0060] In addition, by analyzing the frequency of exceptions and the fluctuation of resource data in the exception list, we can focus on the dependency relationship of various resources when the exception occurs. Based on the predefined resource dependency chain model in the task fingerprint library, we can extract the synergy pattern of multiple resources from the abnormal event, such as network delays causing slow data reading and writing, which in turn affects the increase in CPU load; by matching the abnormal resource dependency chain in the task fingerprint library, we can identify the root cause of the exception, which may be a series of resource bottlenecks caused by network problems, or performance degradation caused by resource contention; based on the results of task fingerprint matching, we can get the root cause analysis results. This task fingerprint-based matching method greatly improves the efficiency and accuracy of root cause analysis, especially when dealing with large-scale distributed tasks. Through the rapid retrieval of the fingerprint library, it can help the system quickly find the historical root causes of similar tasks, provide reference solutions, and reduce the time and workload of manual analysis.

[0061] The embodiment of the present invention obtains the operation log and resource data of the task; uses a pre-configured rule engine to extract the exception list of the task from the operation log; based on the exception list and resource data, uses a pre-built task fingerprint library to determine the root cause analysis result of the task; thereby, the time cost of manual processing can be reduced, the processing efficiency of the root cause analysis is improved, and the accuracy of the root cause analysis result is improved.

[0062] Optionally, based on the exception list and resource data, a pre-built task fingerprint library is used to determine the root cause analysis result of the task, including: standardizing the exception list and resource data respectively to generate exception feature text and data feature text; wherein the standardization includes formatting and normalization; splicing the exception feature text and the data feature text, and determining the fingerprint string of the task based on the spliced ​​feature text; and determining the root cause analysis result of the task based on the fingerprint string of the task and the pre-built task fingerprint library.

[0063] In the root cause analysis process, the task exception list and resource data are first standardized, and the exception feature text and data feature text are extracted from the standardized exception list and resource data. The standardization of the exception list includes formatting, etc., that is, the extracted exception information is uniformly structured according to predefined rules, so that the exception information of different tasks has a consistent representation method. The standardization of resource data includes normalization operations, etc., that is, various resource indicators (such as memory usage, CPU consumption, execution time, etc.) are scaled within a certain range or ratio to ensure that different indicators can be directly compared.

[0064] The generated abnormal feature text and data feature text are spliced ​​together to form a complete feature text containing abnormal and resource information. The fingerprint string of the task is calculated based on the spliced ​​feature text. This fingerprint string is usually generated by algorithms such as SimHash. It is a hash value of fixed length, that is, a simplified representation of the abnormal information and resource data of the task, which is convenient for quick comparison and matching. Based on the generated task fingerprint string, it can be matched in the pre-built task fingerprint library. By comparing the similarity between the fingerprint string of the new task and the existing fingerprints in the library, the historical task closest to the new task characteristics can be quickly found. If a similar fingerprint is found, the system will directly return the root cause analysis result corresponding to the fingerprint, thereby providing possible root causes and corresponding solution suggestions for the new task; if there is no matching fingerprint, further in-depth analysis will be performed to generate new root cause analysis results and add them to the fingerprint library.

[0065] Optionally, determining the root cause analysis result of the task based on the fingerprint string of the task and a pre-built task fingerprint library includes: traversing the fingerprint strings in the task fingerprint library to determine a similar fingerprint string to the fingerprint string of the task; in response to determining a similar fingerprint string to the fingerprint string of the task, determining the root cause analysis result corresponding to the similar fingerprint string as the root cause analysis result of the task; in response to not determining a similar fingerprint string to the fingerprint string of the task, determining the exception class corresponding to the exception list and the weight of each exception class based on a pre-built exception rule library, and determining the root cause analysis result of the task according to the exception class corresponding to the exception list and the weight of each exception class.

[0066] Specifically, all fingerprint strings in the task fingerprint library are traversed, and the fingerprint string of the task is compared with the existing fingerprint strings in the library one by one. The potential similar fingerprint strings are determined by calculating the similarity between them. The similarity can be achieved by calculating the Hamming distance of the fingerprint string or other methods. The smaller the Hamming distance, the more similar the fingerprints are. If it is determined that the fingerprint string of the task has sufficient similarity with the fingerprint string in a fingerprint library, it is considered that a similar fingerprint string has been found. At this time, the historical root cause analysis result corresponding to the similar fingerprint string can be directly returned as the root cause analysis result of the current task. This method can quickly determine the root cause of the task, which is particularly suitable for task scenarios that are highly similar to historical abnormal situations, thereby improving the efficiency of root cause analysis. If a similar fingerprint string cannot be found, that is, the fingerprint string of the task does not have a sufficiently close match with the fingerprint library, further analysis can be performed; the task's exception list can be analyzed based on the pre-built exception rule library, which contains the definitions, weights, and classification standards of different exception classes, etc. According to the exception rule library, each exception item in the exception list is classified, its corresponding exception class is determined, and the weight of each exception class is calculated. The weight represents the importance or influence of the exception class in the entire exception list; based on the exception class and the weight of each exception class, the root cause analysis result of the task is finally determined through comprehensive calculation, for example, the exception class with a higher weight is regarded as the main cause of the task problem, and the root cause analysis result and corresponding suggestions are output accordingly. If the system generates a new root cause result after this calculation process, the result can also be used to update the task fingerprint library in the future, providing a reference for similar tasks in the future, thereby greatly reducing the cost of manual maintenance.

[0067] Optionally, determining a similar fingerprint string to the fingerprint string of the task includes: performing a Hamming distance calculation on the fingerprint string of the task and a fingerprint string in a task fingerprint library to obtain a Hamming distance value; in response to the Hamming distance value being less than a preset Hamming distance threshold, determining the fingerprint string in the task fingerprint library as a similar fingerprint string to the fingerprint string of the task.

[0068] In the process of determining the fingerprint string similar to the task, the fingerprint string generated by the current task is compared with all the existing fingerprint strings in the task fingerprint library. This can be achieved by calculating the Hamming distance. The Hamming distance refers to the bit-by-bit comparison of the number of different characters between two strings of the same length. The comparison of the fingerprint string of the task with the fingerprint string in the task fingerprint library can be achieved by writing a corresponding algorithm to calculate the Hamming distance and obtain the Hamming distance value between each pair of fingerprint strings. The calculated Hamming distance value is compared with the preset Hamming distance threshold. If the Hamming distance value is less than the preset threshold, it means that the fingerprint string of the current task has a high similarity in characteristics with a fingerprint string in the task fingerprint library. This qualified fingerprint string is determined as a similar fingerprint string of the current task fingerprint string. It can efficiently identify abnormal situations with similar characteristics to the current task in history, so as to quickly obtain relevant root cause analysis results. The identification of similar fingerprint strings can not only help operation and maintenance personnel understand similar problems encountered in the past, but also provide an effective basis for rapid troubleshooting, further improving the operation and maintenance efficiency of the system.

[0069] When determining similar fingerprint strings of a task's fingerprint string, in addition to the Hamming distance, other similarity measurement methods can also be used. For example, cosine similarity is used to measure similarity by calculating the cosine value of the angle between two fingerprint strings as vectors. This can effectively process high-dimensional data. The closer the value is to 1, the higher the similarity between the fingerprint strings. In addition, the Jaccard similarity coefficient can be used to evaluate the degree of overlap between two fingerprint strings in the form of a set by calculating the ratio of their intersection to their union. The higher the value, the stronger the similarity.

[0070] Optionally, after determining the root cause analysis result of the task according to the exception class corresponding to the exception list and the weight of each exception class, it also includes: sending the root cause analysis result of the task to the user and receiving the user's feedback result; in response to the feedback result being satisfactory, storing the fingerprint string and the root cause analysis result of the task in the fingerprint library; in response to the feedback result being unsatisfactory, repeating the following steps until the number of unsatisfactory feedback reaches a preset feedback threshold or the feedback result is satisfactory again: adjusting the weight of the exception class using a pre-configured optimization strategy, redetermining the root cause analysis result of the task according to the adjusted weight of the exception class, sending the redetermined root cause analysis result of the task to the user again, and receiving the user's feedback result again.

[0071] After determining the root cause analysis result of the task, it is sent to the user to obtain feedback. The feedback mechanism can improve the effectiveness and accuracy of the root cause analysis result. When the user is satisfied with the root cause analysis result, the fingerprint string and the root cause analysis result of the task are stored in the fingerprint library so that similar tasks can be quickly identified and analyzed in the future, and the subsequent root cause analysis can be continuously optimized. If the user is dissatisfied with the analysis result, a feedback loop can be started. In this process, the weight of the abnormal class is adjusted by using the pre-configured optimization strategy, such as increasing the abnormal class with a higher weight or reducing the abnormal class with a lower weight, etc. After adjustment, the root cause analysis result of the task is re-determined; the new root cause analysis result is sent to the user again to obtain the user's feedback again. If the user is satisfied with the new root cause analysis result, the process ends and the new root cause analysis result is stored; if the user is still dissatisfied, the process continues until the number of "dissatisfied" feedback from the user reaches the preset feedback threshold, or the user finally expresses satisfaction with the adjusted result. Through this continuous feedback and adjustment mechanism, it can effectively adapt to user needs, improve the accuracy and reliability of root cause analysis, and maintain efficient operation and maintenance management in a constantly changing environment.

[0072] Optionally, the method also includes: in response to the number of unsatisfactory feedbacks reaching a preset feedback threshold, sending the operation log and resource data of the task to the operation and maintenance personnel, and receiving the root cause analysis results of the task by the operation and maintenance personnel, and determining the root cause analysis results of the task by the operation and maintenance personnel as the root cause analysis results of the task.

[0073] When the number of dissatisfied user feedbacks reaches a preset feedback threshold, predefined countermeasures can be initiated, and the task's operation log and resource data can be collated and sent to the operation and maintenance personnel. After receiving this information, the operation and maintenance personnel can conduct a detailed review of the task's operation log and resource data, and further explore and evaluate the root cause analysis results based on their experience and expertise. The operation and maintenance personnel may identify abnormal patterns or potential problems that the user has not noticed, so that they can more accurately determine the root cause of the problem. Based on these analyses, the operation and maintenance personnel form their own root cause analysis results. The root cause analysis results provided by the operation and maintenance personnel are received and determined as the official root cause analysis results of the task. The embodiment of the present invention not only enhances the analysis capability of the system, but also improves the efficiency and accuracy of problem solving by introducing the professional perspective of the operation and maintenance personnel. This collaborative working mode forms an effective feedback loop between the automation system and manual intervention, further improving the quality of operation and maintenance management.

[0074] Figure 2 It is a schematic diagram of the main links of the root cause analysis of big data anomaly tasks according to a preferred embodiment of the present invention. Figure 3 FIG. 1 is a schematic diagram of the main process of root cause analysis of big data anomaly tasks according to a preferred embodiment of the present invention. Figure 2 and Figure 3 As shown, the embodiment of the present invention is based on a constructed exception rule library, which contains basic elements such as common exception items, exception classes to which the exception items belong, and general solutions to exception classes. The construction cost of this exception rule library is low and it is applicable to all big data platforms based on Spark / Hive. The exception rule library covers common exceptions in big data tasks and sets the weight of the exception class to which the rules belong. For example, for the exception of the OutOfMemory (insufficient memory), it is further divided into two subclasses: DriverOOM (driver memory overflow error) and ExecutorOOM (executor memory overflow error), and equal weights (0.5 and 0.5) are assigned to these two subclasses respectively. That is to say, the OutOfMemory exception may be caused by DriverOOM or ExecutorOOM, and the possibility of each case is equal. The exception rule library also includes general solutions for exception classes. For example, for the ExecutorOOM exception, the recommended solution is to increase the memory configuration of the executor. The specific configuration method is to set the memory size of each executor in Spark to 1.3 times the original value, which can be achieved through conf spark.executor.memory=source value*1.3. This adjustment can solve the problem caused by insufficient memory by increasing the executor memory. The matching rules for each exception item are defined in the rule engine. Each exception class also has a badCount field, which records the number of times the exception class is invalid. The initial value is 0. The rule engine can obtain exception information from the running log.

[0075] Specifically, this embodiment implements root cause analysis and rule optimization of task exceptions through fingerprint calculation, root cause analysis, and root cause update. First, obtain the task's operation log and resource data by subscribing to Yarn logs or reading JobHistory. Use the rule engine to extract the task's exception list from the operation log, and determine the exception class and its weight corresponding to each exception item according to the weight in the exception rule library, for example, exception 1: Exce_a 0.4, Exce_b 0.5, Exce_c 0.1; exception 2: Exce_a 0.5, Exce_b 0.5; exception 3: Exce_b 0.6, Exce_c 0.4. Then the task fingerprint is calculated. The exception items are matched from the task log by the rule engine. In order to unify the format, the exception list is formatted. If there are less than five exception items, they are filled to obtain the exception feature text; the resource data (such as memory usage, running time, GC time, etc.) is normalized and spliced ​​into data feature text; the exception feature text is spliced ​​with the data feature text, and the SimHash algorithm is used to calculate the 64-bit fingerprint string of the task. The Hamming distance between the fingerprint string and the existing fingerprints in the fingerprint library is calculated. If the Hamming distance k is less than the set Hamming distance threshold p, it is considered similar and the matching root cause analysis result is directly returned.

[0076] If there is no similar fingerprint in the fingerprint library, the root cause analysis is performed according to the abnormal class in the abnormal list. By calculating the weight of each abnormal class, the abnormal class with the highest weight is determined as the root cause of the task and corresponding suggestions are provided. Users or operation and maintenance personnel can provide feedback on the root cause analysis results of the system. If the feedback is satisfactory, the fingerprint string of the task and the root cause analysis results are stored in the task fingerprint library so that similar tasks in the future can quickly match the root cause. If the feedback is unsatisfactory, the badCount value is increased. When badCount does not reach the set unsatisfactory feedback threshold m, the system will self-optimize the weight, select the abnormal class with the second highest weight as the new root cause, and adjust the weight of the abnormal class according to the set iteration ratio, and repeatedly optimize. During the continuous optimization process, if the task abnormality still cannot be solved and badCount reaches the set unsatisfactory feedback threshold, the system will determine that the abnormality of the task is a new situation, and the operation and maintenance personnel need to intervene to manually analyze the abnormality. After the operation and maintenance personnel solve the problem, the system will maintain the new situation and its root cause analysis results in the abnormal rule library to enhance the coverage of the rules and improve the automation level of abnormality handling. For example, assume that the weight calculation result is: Exce_a 30%, Exce_b 53.33%, Exce_c 16.67%; the system returns Exce_b with the highest weight as the root cause and provides a corresponding solution; after obtaining feedback from users or operation and maintenance personnel, if the feedback is satisfactory, the task fingerprint and the root cause are stored in the fingerprint library to facilitate the matching of subsequent tasks. If the feedback is unsatisfactory, badCount is increased. When badCount is less than m, the abnormal class with the second highest weight (such as Exce_a) is selected for self-optimization, and the weight is adjusted according to α (iteration ratio, assumed to be 1%); assuming that there are three abnormal items in the task, the optimized weights are abnormality 1: Exce_a 0.55, Exce_b 0.425, Exce_c 0.025; abnormality 2: Exce_a 0.65, Exce_b 0.35; abnormality 3: Exce_a 0.15, Exce_b 0.525, Exce_c 0.325; the adjusted weights are Exce_a 45%, Exce_b 43.3%, and Exce_c 11.7%; output root causes and suggestions according to the adjusted weights, and continuously optimize the abnormality ratio until badCount>=m, which means that the abnormality of this task is a new situation, and operation and maintenance intervention is required to manually analyze the abnormal situation and supplement the maintenance rule base; output the root cause according to the new situation maintained by operation and maintenance, and store the fingerprint in the database.

[0077] The embodiment of the present invention can quickly locate the root cause of task exceptions through automated task log analysis and fingerprint matching mechanisms, reducing the time for manual step-by-step troubleshooting and significantly improving the efficiency of exception handling. The self-optimization mechanism allows the system to gradually adjust the weights of exception classes when the root cause analysis results are unsatisfactory, so that the accuracy of the analysis results continues to improve with the optimization process, thereby more effectively handling subsequent exceptions and reducing the number of interventions by operation and maintenance personnel. Manual intervention is only required when complex or new problems arise, greatly reducing the burden of operation and maintenance. At the same time, it has the ability to dynamically update the rule base. With the feedback from users or operation and maintenance personnel during task processing, new abnormal situations can be maintained and stored in a timely manner, so that the system can quickly respond to similar problems in future tasks, improving the coverage and flexibility of exception handling.

[0078] According to a second aspect of an embodiment of the present invention, a root cause analysis device is provided.

[0079] Figure 4 FIG. 1 is a schematic diagram of the main modules of the root cause analysis device according to an embodiment of the present invention. Figure 4 As shown, a root cause analysis device 400 includes:

[0080] The acquisition module 401 is used to acquire the operation log and resource data of the task;

[0081] An extraction module 402 is used to extract an exception list of tasks from the operation log using a pre-configured rule engine;

[0082] The determination module 403 is used to determine the root cause analysis result of the task based on the exception list and resource data using the pre-built task fingerprint library.

[0083] Optionally, the determination module 403 is further configured to:

[0084] The exception list and resource data are standardized respectively to generate an exception feature text and a data feature text; wherein the standardization process includes formatting process and normalization process;

[0085] The abnormal feature text and the data feature text are spliced ​​together, and the fingerprint string of the task is determined according to the spliced ​​feature text;

[0086] The root cause analysis result of the task is determined based on the fingerprint string of the task and a pre-built task fingerprint library.

[0087] Optionally, the determination module 403 is further configured to:

[0088] Traverse the fingerprint strings in the task fingerprint library to determine the fingerprint strings similar to the fingerprint string of the task;

[0089] In response to determining a similar fingerprint string to the fingerprint string of the task, determining a root cause analysis result corresponding to the similar fingerprint string as the root cause analysis result of the task;

[0090] In response to a fingerprint string similar to the fingerprint string of the task that has not been determined, the exception class corresponding to the exception list and the weight of each exception class are determined based on a pre-built exception rule library, and the root cause analysis result of the task is determined according to the exception class corresponding to the exception list and the weight of each exception class.

[0091] Optionally, the determination module 403 is further configured to:

[0092] Calculate the Hamming distance between the fingerprint string of the task and the fingerprint string in the task fingerprint library to obtain the Hamming distance value;

[0093] In response to the Hamming distance value being less than a preset Hamming distance threshold, the fingerprint string in the task fingerprint library is determined as a similar fingerprint string to the fingerprint string of the task.

[0094] Optionally, the root cause analysis device 400 further includes a feedback module, and the feedback module is used to:

[0095] Send the root cause analysis results of the task to the user and receive feedback from the user;

[0096] In response to the feedback result being satisfactory, storing the fingerprint string and the root cause analysis result of the task in the fingerprint library;

[0097] In response to the feedback result being unsatisfactory, the following steps are repeated until the number of unsatisfactory feedbacks reaches a preset feedback threshold or the feedback result is satisfactory again: the weight of the exception class is adjusted using a pre-configured optimization strategy, the root cause analysis result of the task is redetermined according to the adjusted weight of the exception class, the root cause analysis result of the redetermined task is sent to the user again, and the user's feedback result is received again.

[0098] Optionally, the root cause analysis device 400 further includes a maintenance module, and the maintenance module is used to:

[0099] In response to the number of unsatisfactory feedbacks reaching a preset feedback threshold, the operation log and resource data of the task are sent to the operation and maintenance personnel, and the root cause analysis result of the task by the operation and maintenance personnel is received, and the root cause analysis result of the task by the operation and maintenance personnel is determined as the root cause analysis result of the task.

[0100] It should be noted that the specific implementation content of the root cause analysis device of the present invention has been described in detail in the above root cause analysis method, so the repeated content will not be described again here.

[0101] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the first aspect of the embodiment of the present invention.

[0102] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0103] According to a fifth aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0104] Figure 5 An exemplary system architecture 500 is shown to which the root cause analysis method or the root cause analysis device according to the embodiment of the present application can be applied.

[0105] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0106] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0107] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0108] The server 505 may be a server that provides various services, such as a background management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 501, 502, and 503. The background management server may analyze and process the received root cause analysis request and other data, and feed back the processing result (such as the root cause analysis result - only an example) to the terminal device.

[0109] It should be noted that the root cause analysis method provided in the embodiment of the present application is generally executed by the server 505 , and accordingly, the root cause analysis device is generally disposed in the server 505 .

[0110] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0111] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present application. Figure 6 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0112] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0113] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0114] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0115] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage media may include, for example, but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The units involved in the embodiments described in the present application may be implemented by software or by hardware. The units described may also be arranged in a processor, for example, it may be described as: a processor includes an acquisition module, an extraction module and a determination module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the acquisition module may be described as a "module for acquiring the operation log and resource data of the task".

[0118] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining the operation log and resource data of the task; extracting the exception list of the task from the operation log using a pre-configured rule engine; and determining the root cause analysis result of the task based on the exception list and resource data using a pre-built task fingerprint library.

[0119] The computer program product provided by the embodiment of the present invention includes a computer program. When the computer program is executed by a processor, the root cause analysis method in the embodiment of the present invention is implemented.

[0120] The technical solution according to the embodiment of the present invention has the following advantages or beneficial effects: by obtaining the operation log and resource data of the task; using a pre-configured rule engine to extract the exception list of the task from the operation log; based on the exception list and resource data, using a pre-built task fingerprint library to determine the root cause analysis result of the task; thereby, the time cost of manual processing can be reduced, the processing efficiency of the root cause analysis can be improved, and the accuracy of the root cause analysis result can be improved.

[0121] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

[0122] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

Claims

1. A root cause analysis method, characterized in that: include: Get the task's running log and resource data; Extracting an exception list of the task from the operation log using a pre-configured rule engine; Based on the exception list and the resource data, a root cause analysis result of the task is determined using a pre-built task fingerprint library.

2. The method according to claim 1, characterized in that: Based on the exception list and the resource data, a root cause analysis result of the task is determined using a pre-built task fingerprint library, including: The abnormal list and the resource data are respectively subjected to standardization processing to generate abnormal feature text and data feature text; wherein the standardization processing includes formatting processing and normalization processing; splicing the abnormal feature text and the data feature text, and determining the fingerprint string of the task according to the spliced ​​feature text; A root cause analysis result of the task is determined based on the fingerprint string of the task and a pre-built task fingerprint library.

3. The method according to claim 2, characterized in that Determining a root cause analysis result of the task based on the fingerprint string of the task and a pre-built task fingerprint library includes: Traversing the fingerprint strings in the task fingerprint library to determine a fingerprint string similar to the fingerprint string of the task; In response to determining a similar fingerprint string to the fingerprint string of the task, determining a root cause analysis result corresponding to the similar fingerprint string as the root cause analysis result of the task; In response to a fingerprint string similar to the fingerprint string of the task not being determined, the exception class corresponding to the exception list and the weight of each exception class are determined based on a pre-built exception rule library, and the root cause analysis result of the task is determined according to the exception class corresponding to the exception list and the weight of each exception class.

4. The method according to claim 3, characterized in that Determining a fingerprint string similar to the fingerprint string of the task includes: Performing Hamming distance calculation on the fingerprint string of the task and the fingerprint string in the task fingerprint library to obtain a Hamming distance value; In response to the Hamming distance value being less than a preset Hamming distance threshold, the fingerprint string in the task fingerprint library is determined as a similar fingerprint string to the fingerprint string of the task.

5. The method according to claim 3, characterized in that: After determining the root cause analysis result of the task according to the exception class corresponding to the exception list and the weight of each exception class, the method further includes: Sending the root cause analysis result of the task to the user and receiving the user's feedback result; In response to the feedback result being satisfactory, storing the fingerprint string and the root cause analysis result of the task in the fingerprint library; In response to the feedback result being unsatisfactory, the following steps are repeated until the number of unsatisfactory feedbacks reaches a preset feedback threshold or the feedback result is satisfactory again: the weight of the exception class is adjusted using a pre-configured optimization strategy, the root cause analysis result of the task is re-determined according to the adjusted weight of the exception class, the re-determined root cause analysis result of the task is sent to the user again, and the user's feedback result is received again.

6. The method according to claim 5, characterized in that The method further comprises: In response to the number of unsatisfactory feedbacks reaching a preset feedback threshold, the operation log and resource data of the task are sent to the operation and maintenance personnel, and the root cause analysis result of the task by the operation and maintenance personnel is received, and the root cause analysis result of the task by the operation and maintenance personnel is determined as the root cause analysis result of the task.

7. A root cause analysis device, characterized in that: include: The acquisition module is used to obtain the task's running log and resource data; An extraction module, used to extract an exception list of the task from the operation log using a pre-configured rule engine; A determination module is used to determine the root cause analysis result of the task based on the exception list and the resource data using a pre-built task fingerprint library.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images