Error log processing method and device, equipment, storage medium and program product

Abstract

The invention provides an error log processing method and device, equipment, a storage medium and a program product, and the error log processing method comprises the steps: obtaining a plurality of error logs, and carrying out the clustering processing of the plurality of error logs through a clustering algorithm, and obtaining a plurality of error log classes; performing filtering processing on each error log class to obtain a target error log class; grouping the error logs in the target error log class to obtain a plurality of first log groups; based on the error logs in the first log groups, error log templates corresponding to the first log groups are generated, and fault causes are analyzed according to the error log templates. According to the method, similar error logs are clustered into one class, so that the rapid detection and analysis capability of the error logs is improved, and the processing efficiency of the error logs is improved. Meanwhile, useless error logs are filtered out, and the accuracy of subsequent error log template generation is improved.

Description

Technical Field

[0001] The present invention relates to the field of operation and maintenance technology, and in particular to an error log processing method, device, equipment, storage medium and program product. Background Art

[0002] With the continuous development of the Internet, application systems are becoming increasingly large and complex, accompanied by an explosive growth in error logs. Faced with massive error logs, the analysis and processing capabilities of relevant operation and maintenance personnel are limited. If the operation and maintenance personnel only rely on analyzing the error logs one by one, it will take a long time, which will result in the corresponding error problems cannot be solved in time, resulting in reduced operation and maintenance efficiency.

[0003] At present, the cause of the failure is determined by searching for frequent error words in the acquired log data set. The reason for adopting such data processing method is that the words that appear in the frequent error reports are largely the root cause of the generation of the error log. Therefore, in the related art, frequent words are first identified, and then different log sets are formed according to the frequent words contained in each log message, and finally a log template is created for each set. However, although this method of creating log templates for different logs by identifying frequent words has high accuracy, it requires traversing all logs to find words, which is time-consuming and inefficient. Summary of the invention

[0004] The present invention provides an error log processing method, device, equipment, storage medium and program product, which are used to solve the defect of low error log processing efficiency in the prior art and improve the error log processing efficiency.

[0005] The present invention provides an error log processing method, comprising: Acquire multiple error logs, and cluster the multiple error logs using a clustering algorithm to obtain multiple error log classes; Filtering each of the error log classes to obtain a target error log class; performing grouping processing on each of the error logs in the target error log class to obtain a plurality of first log groups; Based on the error logs in each of the first log groups, an error log template corresponding to each of the first log groups is generated, and the cause of the failure is analyzed according to the error log template.

[0006] According to an error log processing method provided by the present invention, clustering the plurality of error logs by a clustering algorithm to obtain a plurality of error log classes includes: Extracting log features of each of the error logs, and converting the log features into numerical feature vectors; Based on the message length, each of the converted error logs is grouped to obtain a plurality of second log groups, and each of the error logs in the second log group is structured; The error logs in each of the second log groups are clustered by using a clustering algorithm to obtain a plurality of error log classes.

[0007] According to an error log processing method provided by the present invention, clustering is performed on each of the error logs in each of the second log groups by a clustering algorithm to obtain a plurality of error log classes, including: In the second log group, calculating the weighted edit distance between any two of the error logs; Based on the weighted edit distance, the error logs in each of the second log groups are clustered by the clustering algorithm to obtain a plurality of error log classes.

[0008] According to an error log processing method provided by the present invention, filtering each error log class to obtain a target error log class includes: Extracting keywords from each of the error logs; Based on the keywords, determining target keywords for each of the error log categories; In each of the error log classes, filtering processing is performed according to the keyword and the target keyword to obtain the target error log class.

[0009] According to an error log processing method provided by the present invention, the error logs in the target error log class are grouped to obtain a plurality of first log groups, including: Based on a preset grouping rule, each of the error logs in the target error log class is grouped to obtain a plurality of first log groups; wherein the grouping rule includes at least one of a timestamp, an event type, a preset keyword, an identifier, and a location.

[0010] According to an error log processing method provided by the present invention, the generating of the error log template corresponding to each of the first log groups includes: Extracting a marker variable of the error log in each of the first log groups; The tag variable is represented by a wildcard to obtain the error log template.

[0011] The present invention also provides an error log processing device, comprising: A clustering processing module is configured to obtain a plurality of error logs and perform clustering processing on the plurality of error logs by a clustering algorithm to obtain a plurality of error log classes; A filtering processing module, configured to filter each of the error log classes to obtain a target error log class; A group processing module, configured to perform group processing on each of the error logs in the target error log class to obtain a plurality of first log groups; The analysis module is configured to generate an error log template corresponding to each of the first log groups based on the error logs in each of the first log groups, and analyze the cause of the failure according to the error log template.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the error log processing methods described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the error log processing method described in any one of the above methods is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned error log processing methods.

[0015] The error log processing method, device, equipment, storage medium and program product provided by the present invention use a clustering algorithm to perform clustering processing on multiple error logs, cluster similar error logs into one category, improve the ability to quickly detect and analyze error logs, and improve the processing efficiency of error logs. Perform filtering processing operations on the clustered error log class to filter out useless error logs, improve the accuracy of subsequent error log template generation, and thus improve the accuracy of fault cause analysis, and then further group the target error log class to obtain multiple first log groups, and generate a corresponding error log template for each first log group, thereby improving the generation speed of the error log template, thereby improving the fault analysis processing efficiency and accelerating the operation and maintenance process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of the error log processing method provided by the present invention.

[0018] Figure 2It is a structural schematic diagram of the error log processing device provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figure 1-Figure 3 The error log processing method, device, equipment, storage medium and program product of the present invention are described.

[0022] Figure 1 FIG. 1 is a flowchart of an error log processing method according to an exemplary embodiment. Figure 1 As shown, in an exemplary embodiment, the error log processing method includes steps 110 to 140, which are described in detail as follows.

[0023] Step 110: Acquire multiple error logs, and cluster the multiple error logs using a clustering algorithm to obtain multiple error log classes.

[0024] In the embodiment of the present invention, the error log is a log file that records the error information of the application or system. When an error or anomaly occurs in the application or system, the relevant error information is usually recorded in the error log so that the developer or system administrator can troubleshoot and repair the fault. The error log usually contains various information when the fault occurs, such as the error type, error code, error information, timestamp, etc. By analyzing the error log, the operation status of the application or system can be understood, potential fault problems can be discovered, and fault troubleshooting and repair can be performed.

[0025] The recording method and format of error logs can vary depending on the application or system. Generally speaking, error logs can be stored in the form of text files, database records, system event logs, etc. In large-scale distributed systems, error logs can be centrally managed through log collection and analysis systems to quickly discover and solve fault problems.

[0026] Since different software systems generate different error logs, the following methods are provided to facilitate error log acquisition:

[0027] Many applications include built-in logging capabilities that can be configured in the application to record errors and events. These logs usually contain detailed information about the error, such as timestamp, error type, error code, stack trace, etc. During the operation of the application, error information may be output to the console or log file. Therefore, for the error log of the application, you can obtain the error log by viewing the output information.

[0028] The operating system usually also records system-level errors and events, such as insufficient memory, hardware failure, service crash, etc. In Windows operating systems, you can use Event Viewer to view system logs to obtain error logs; in Linux operating systems, you can use syslog or journalctl tools to view system logs to obtain error logs.

[0029] Many application performance monitoring tools can help track errors and events in applications. For example, AppDynamics, New Relic, Dynatrace, Splunk, etc. can help capture errors in applications and provide detailed information and alerts about the errors, while error monitoring tools can automatically collect and analyze application error logs, such as Sentry, Bugsnag, etc. Therefore, error logs can be obtained through error monitoring tools.

[0030] Network devices, such as routers, switches, and firewalls, also log errors and events. You can use device management tools to obtain error logs to understand errors and events that occur in the network.

[0031] Some debugging tools can help developers capture error information in applications, such as VisualStudio, Eclipse, etc. Therefore, error logs can be obtained through debugging tools.

[0032] Some applications may send error logs to a remote logging system. Therefore, you can obtain the error logs by accessing the remote logging system.

[0033] Multiple ways of obtaining error logs can provide stronger compatibility when connecting to different software systems.

[0034] Since the number of error logs obtained is relatively large, if they are analyzed one by one, it will generate a lot of computing power and take a long time, resulting in low efficiency in the subsequent analysis of the cause of the failure. Therefore, the embodiment of the present invention adopts a clustering algorithm to cluster the massive error logs to obtain multiple error log classes, each of which includes one or more error logs.

[0035] Step 120: Filter each of the error log classes to obtain a target error log class.

[0036] In the embodiment of the present invention, a clustering algorithm is used to cluster similar error logs into one category. However, after clustering, error logs of the same category may still be generated by different message templates. Therefore, it is necessary to further filter the error logs in each category to filter out useless error logs in each category, so as to improve the accuracy of subsequent error log template generation.

[0037] Step 130: group the error logs in the target error log class to obtain a plurality of first log groups.

[0038] In the embodiment of the present invention, in each target error log class, each error log is further grouped to obtain a plurality of first log groups.

[0039] Step 140: Generate an error log template corresponding to each of the first log groups based on the error logs in each of the first log groups, and analyze the cause of the failure according to the error log template.

[0040] In the embodiment of the present invention, according to the error logs in the obtained multiple first log groups, an error log template corresponding to each first log group is generated, and the error log template can more efficiently handle errors in a large number of error logs. A fast and standardized error handling process is provided for operation and maintenance personnel, and operation and maintenance personnel are helped to solve error problems more efficiently, reducing the time and cost of error repair.

[0041] In an embodiment of the present invention, an error log template corresponds to a fault cause, and these fault causes can be sorted in a certain order, such as by the time of occurrence of the fault, to obtain a fault cause list, which can clearly display the possible fault causes so that operation and maintenance personnel can analyze the fault causes based on the table.

[0042] In the embodiment of the present invention, a clustering algorithm is used to perform clustering processing on multiple error logs, and similar error logs are clustered into one category, thereby improving the ability to quickly detect and analyze error logs. The clustered error log class is filtered to filter out useless error logs, improve the accuracy of subsequent error log template generation, and thus improve the accuracy of fault cause analysis, and then further group the target error log class to obtain multiple first log groups, and generate a corresponding error log template for each first log group, thereby improving the speed of generating the error log template, thereby improving the efficiency of fault analysis and processing, and accelerating the operation and maintenance process.

[0043] In an exemplary embodiment of the present invention, clustering the plurality of error logs by a clustering algorithm to obtain a plurality of error log classes includes the following steps, which are described in detail as follows.

[0044] Extract log features of each of the error logs, and convert the log features into numerical feature vectors.

[0045] Based on the message length, each of the converted error logs is grouped to obtain a plurality of second log groups, and each of the error logs in the second log group is structured.

[0046] The error logs in each of the second log groups are clustered by using a clustering algorithm to obtain a plurality of error log classes.

[0047] In the embodiment of the present invention, since the clustering algorithm involves a large number of similarity calculations, the error log is processed before clustering to reduce the time complexity. Specifically, key log features, such as timestamp, log level, message content, etc., are extracted from the error log and converted into numerical feature vectors for clustering calculation. It can be understood that preprocessing the error log can reduce the time complexity of clustering and help improve the operating efficiency of the overall algorithm.

[0048] Some marker variables in error logs usually appear in the form of numbers, URIs, IP addresses, etc., or these marker variables contain curly brackets, square brackets, parentheses, or underscores, slashes, backslashes, etc. These marker variables are very easy to identify in event messages and can be identified using rule definition methods. Therefore, these typical marker variables are described by defining explicit rule expressions, and these marker variables are represented by wildcards or other placeholders. After this step, the remaining markers are candidate marker constants.

[0049] Specifically, each time an error log is read, the total number of tokens in the error log is recorded, and the total number of tokens is used as the corresponding message length. Each token is compared with the pre-defined rules, and the tokens that meet the rules are replaced with wildcards or other placeholders for subsequent processing and analysis, which helps to reduce noise and improve the accuracy of the clustering algorithm.

[0050] The message length of each error log is counted, and the error logs with the same message length are divided into a group to obtain multiple second log groups. In the same second log group, a clustering algorithm is used to cluster similar error logs in the same group into one category to obtain multiple error log categories. By analyzing a large number of error logs, it is found that logs generated by the same message template are usually similar, which is related to the fact that they all have the same tag constant. The main idea of ​​the clustering algorithm is to cluster similar error logs into one category by comparing the similarities between error logs.

[0051] In an exemplary embodiment of the present invention, clustering the error logs in the second log groups by a clustering algorithm to obtain a plurality of error log classes includes the following steps, which are described in detail below.

[0052] In the second log group, a weighted edit distance between any two of the error logs is calculated.

[0053] Based on the weighted edit distance, the error logs in each of the second log groups are clustered by the clustering algorithm to obtain a plurality of error log classes.

[0054] In the embodiment of the present invention, a clustering algorithm is used in each second log group obtained above to cluster similar error logs into one category. The distance measurement is more important in clustering. The embodiment of the present invention adopts weighted edit distance, that is, weighted string edit distance. The specific definition of weighted edit distance is as follows.

[0055] .

[0056] Where EO is the number of operations from string S1 to string S2, x i represents the position index of the word in the string at the i-th operation, and v represents the hyperparameter controlling the weight.

[0057] A distance threshold is preset. In the same second log group, for two different error logs, their weighted edit distances are calculated first. A smaller weighted edit distance means that the two error logs are more similar, while a larger distance means that they are more different. If the calculated weighted edit distance is less than the distance threshold, it indicates that the two error logs are connected.

[0058] By calculating the weighted edit distance between each pair of error logs in the second log grouping, we can obtain the set of weighted edit distances of all error log pairs and cluster them using a clustering algorithm (such as K-means, DBSCAN, etc.).

[0059] If k=2 is set, the K-means clustering algorithm is used to divide these distance values ​​into two categories. The k-means algorithm calculates the center value of each category (that is, the average value of all points in the category). Compare the center values ​​of the two categories, select the category with the smaller center value, and find the maximum weighted edit distance value in the category as the category threshold. The category threshold will be used to determine whether new error logs need to be classified into the category with the smaller center value, or whether a new cluster needs to be formed. If the weighted edit distance between a new error log and any error log in the category with the smaller center value is less than or equal to the category threshold, it will be classified into the category with the smaller center value; if it is greater than the category threshold, it may be regarded as the starting point for forming a new cluster.

[0060] In an exemplary embodiment of the present invention, filtering each of the error log classes to obtain a target error log class includes the following steps, which are described in detail below.

[0061] Extracting keywords from each of the error logs; Based on the keywords, target keywords of each of the error log categories are determined.

[0062] In each of the error log classes, filtering processing is performed according to the keyword and the target keyword to obtain the target error log class.

[0063] In the embodiment of the present invention, among the various error log classes after clustering, there may be error logs that do not belong to this class. If the error logs are not eliminated, the accuracy of the subsequent error log template generation will be affected. Therefore, the error log classes obtained by clustering processing will be further analyzed and processed.

[0064] Specifically, obtain the keywords of the error logs in each error log class, calculate the keywords with the largest proportion in each error log class, and use the keywords as the target keywords as the basis for filtering. If the keywords of an error log in a certain error log class do not match the target keywords of this error log class, delete this error log from the error log class and put it in other log modules. Filter all error log classes, and form the corresponding target error log class for the remaining error logs in the error log class.

[0065] By further filtering out error logs, the error logs in each error log category are made more unified, and the accuracy of error log template generation is further improved. In addition, by placing the deleted error logs into other log modules, low-probability fault events can be raised in the subsequent fault opinion generation. This event may not cause the real cause of the fault, but it can still give the operation and maintenance personnel relevant alarm prompts so that the operation and maintenance personnel can perform further operation and maintenance processing based on the fault analysis provided by this system.

[0066] In an exemplary embodiment of the present invention, the grouping process of each error log in the target error log class to obtain a plurality of first log groups includes the following steps, which are described in detail as follows.

[0067] Based on a preset grouping rule, each of the error logs in the target error log class is grouped to obtain a plurality of first log groups; wherein the grouping rule includes at least one of a timestamp, an event type, a preset keyword, an identifier, and a location.

[0068] In the embodiment of the present invention, the error logs in the target error log class are grouped so that each group corresponds to an independent print statement.

[0069] The error logs after filtering are grouped according to certain rules or characteristics. The specific grouping method can be determined according to actual needs and analysis objectives. The following are some common grouping methods: Based on timestamp: Group error logs by timestamp, such as by hour, day, week, etc., so that you can view log activities within a specific time period.

[0070] Based on event type: error logs are grouped according to event type, such as error, warning, information, etc., so that different types of log records can be quickly located.

[0071] Based on preset keywords: Group error logs according to the specific keywords or keyword groups, so that related log records can be classified together for further analysis and processing.

[0072] Based on identifiers: Group error logs by user ID, device ID, or other identifiers, so that you can track activity records for specific users or devices.

[0073] Based on location: Group error logs by geographic location, such as country, city, region, etc., so that you can understand the log activities in different regions.

[0074] In an exemplary embodiment of the present invention, the generating of the error log template corresponding to each of the first log groups includes the following steps, which are described in detail as follows.

[0075] Extract the mark variable of the error log in each of the first log groups.

[0076] The tag variable is represented by a wildcard to obtain the error log template.

[0077] In the embodiment of the present invention, the error log is generally independent text data, and the text content records the error events that occurred in the system, including a message part and some event attributes, and the event attributes include a timestamp, a message level, etc. The message part is defined by the source code, for example: print(message,Connection error from %s port %d,ipadress,portnumber), which can generate the following log data: Connection error from 192.168.10.6 port 22; Connection error from 192.168.10.7 port 80; Connection error from 192.168.10.16 port 21; Connection error from 192.168.10.9 port 25.

[0078] The error logs in a first log group can be represented by an error log template. By using the common part of all error logs in a first log group as a tag constant, and the rest as tag variables, and uniformly representing them with wildcards, these error logs can form an error log template. The tag variable part in the above error log is represented by a wildcard, and the resulting error log template can be represented as: Connection error from * port*.

[0079] In most cases, a log corresponds to a message, a message corresponds to a message template / event type, and an error log template corresponds to a print statement. Log parsing is to extract the error log template from a large amount of original log data, so that an error log corresponds to an event type, and the event sequence is obtained by arranging them in chronological order.

[0080] In an exemplary embodiment of the present invention, since the error logs generated by each software system are different, if the processing system running the error log processing method is not updated and is directly applied to any software system, the accuracy will inevitably decrease over time, so the processing system needs to be updated. Usually, operation and maintenance personnel will perform regular updates, but when the processing system is in a stable state, it is not actually advisable to update frequently, otherwise it will damage the reliability of the processing system. Therefore, this proposal also proposes an update method for determining whether the processing system needs to be updated, which will not damage the reliability of the processing system while improving the accuracy of the processing system fault analysis.

[0081] An update threshold is set in advance. When the update degree of the processing system exceeds the preset update threshold, it indicates that the processing system needs to be updated. After the processing system generates an error log template, the processing system can generate a fault cause based on the error log template and send the fault cause to the operation and maintenance operator. The operation and maintenance personnel can make a judgment based on the fault cause given by the system and feedback an accuracy level to the processing system. The processing system makes a system update judgment based on the previous cluster analysis based on the accuracy level. When the accuracy level feedback from the operation and maintenance personnel is high, it means that the fault analysis accuracy of the processing system is high, and the corresponding update degree can take a lower value, indicating that the expected update degree of the processing system is low. For example, if the operation and maintenance personnel give a score of 9 points for the accuracy of the fault cause given by the processing system (the full score is set to 10 points), the update degree can take 1 point (the update threshold is set to 6). At this time, the update degree does not exceed the update threshold, and the processing system does not need to be updated.

[0082] The embodiment of the present invention uses the update degree to accurately determine whether the processing system needs to be updated, thereby avoiding the problem of model instability caused by frequent updates.

[0083] In an exemplary embodiment of the present invention, a single error log may also be processed, and the processing flow is as follows.

[0084] An error log template is obtained in advance according to the aforementioned data processing method, or according to a preset log format and a known error type. After receiving a single error log, the error log is classified by a classification algorithm or a predefined rule, and the error log is matched with the first log group corresponding to the error log template to determine the corresponding error log template.

[0085] The error log processing device provided by the present invention is described below. The error log processing device described below and the error log processing method described above can be referred to each other. It should be noted that the device provided in the following embodiment and the method provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, which will not be repeated here.

[0086] In an exemplary embodiment of the present invention, see Figure 2 , Figure 2 An error log processing device is shown according to an exemplary embodiment, including the following modules.

[0087] The clustering processing module 210 is configured to obtain a plurality of error logs and perform clustering processing on the plurality of error logs by a clustering algorithm to obtain a plurality of error log classes; A filtering processing module 220 is configured to filter each of the error log classes to obtain a target error log class; The group processing module 230 is configured to perform group processing on each of the error logs in the target error log class to obtain a plurality of first log groups; The analysis module 240 is configured to generate an error log template corresponding to each of the first log groups based on the error logs in each of the first log groups, and analyze the cause of the failure according to the error log template.

[0088] In an exemplary embodiment of the present invention, the clustering processing module 210 includes: A first extraction submodule is configured to extract log features of each of the error logs and convert the log features into a numerical feature vector; A structured processing submodule is configured to group the converted error logs based on the message length to obtain a plurality of second log groups, and perform structured processing on the error logs in the second log groups; The clustering processing submodule is configured to perform clustering processing on each of the error logs in each of the second log groups through a clustering algorithm to obtain a plurality of error log classes.

[0089] In an exemplary embodiment of the present invention, the clustering processing submodule includes: a calculation unit, configured to calculate a weighted edit distance between any two of the error logs in the second log group; The clustering processing unit is configured to perform clustering processing on each of the error logs in each of the second log groups based on the weighted edit distance by using the clustering algorithm to obtain a plurality of the error log classes.

[0090] In an exemplary embodiment of the present invention, the filtering processing module 220 includes: A second extraction submodule is configured to extract keywords from each of the error logs; A determination submodule, configured to determine a target keyword for each of the error log classes based on the keyword; The filtering processing submodule is configured to perform filtering processing in each of the error log classes according to the keyword and the target keyword to obtain the target error log class.

[0091] In an exemplary embodiment of the present invention, the packet processing module 230 includes: The group processing submodule is configured to group the error logs in the target error log class based on preset grouping rules to obtain multiple first log groups; wherein the grouping rules include at least one of a timestamp, an event type, a preset keyword, an identifier and a location.

[0092] In an exemplary embodiment of the present invention, the analysis module 240 includes: A third extraction submodule is configured to extract a marker variable of the error log in each of the first log groups; The representation submodule is configured to represent the tag variable through a wildcard to obtain the error log template.

[0093] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the error log processing method, which includes: obtaining multiple error logs, and clustering the multiple error logs through a clustering algorithm to obtain multiple error log classes; filtering each error log class to obtain a target error log class; grouping each error log in the target error log class to obtain multiple first log groups; based on the error logs in each first log group, generating an error log template corresponding to each first log group, and analyzing the cause of the fault according to the error log template.

[0094] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0095] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the error log processing method provided by the above methods, which method includes: obtaining multiple error logs, and clustering the multiple error logs through a clustering algorithm to obtain multiple error log classes; filtering each of the error log classes to obtain a target error log class; grouping each of the error logs in the target error log class to obtain multiple first log groups; based on the error logs in each of the first log groups, generating error log templates corresponding to each of the first log groups, and analyzing the cause of the fault according to the error log template.

[0096] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the error log processing method provided by the above-mentioned methods, the method comprising: obtaining multiple error logs, and clustering the multiple error logs through a clustering algorithm to obtain multiple error log classes; filtering each of the error log classes to obtain a target error log class; grouping each of the error logs in the target error log class to obtain multiple first log groups; based on the error logs in each of the first log groups, generating an error log template corresponding to each of the first log groups, and analyzing the cause of the fault according to the error log template.

[0097] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing an error log, characterized in that: include: Acquire multiple error logs, and cluster the multiple error logs using a clustering algorithm to obtain multiple error log classes; Filtering each of the error log classes to obtain a target error log class; performing grouping processing on each of the error logs in the target error log class to obtain a plurality of first log groups; Based on the error logs in each of the first log groups, an error log template corresponding to each of the first log groups is generated, and the cause of the failure is analyzed according to the error log template.

2. The error log processing method according to claim 1, characterized in that: The clustering algorithm is used to cluster the plurality of error logs to obtain a plurality of error log classes, including: Extracting log features of each of the error logs, and converting the log features into numerical feature vectors; Based on the message length, each of the converted error logs is grouped to obtain a plurality of second log groups, and each of the error logs in the second log group is structured; The error logs in each of the second log groups are clustered by using a clustering algorithm to obtain a plurality of error log classes.

3. The error log processing method according to claim 2, characterized in that: The clustering algorithm is used to cluster the error logs in the second log groups to obtain a plurality of error log classes, including: In the second log group, calculating the weighted edit distance between any two of the error logs; Based on the weighted edit distance, the error logs in each of the second log groups are clustered by the clustering algorithm to obtain a plurality of error log classes.

4. The error log processing method according to claim 1, characterized in that: The filtering process is performed on each of the error log classes to obtain a target error log class, including: Extracting keywords from each of the error logs; Based on the keywords, determining target keywords for each of the error log categories; In each of the error log classes, filtering processing is performed according to the keyword and the target keyword to obtain the target error log class.

5. The error log processing method according to claim 1, characterized in that: The grouping process of each of the error logs in the target error log class to obtain a plurality of first log groups includes: Based on a preset grouping rule, each of the error logs in the target error log class is grouped to obtain a plurality of first log groups; wherein the grouping rule includes at least one of a timestamp, an event type, a preset keyword, an identifier, and a location.

6. The error log processing method according to any one of claims 1 to 5, characterized in that: The generating the error log template corresponding to each of the first log groups includes: Extracting a marker variable of the error log in each of the first log groups; The tag variable is represented by a wildcard to obtain the error log template.

7. An error log processing device, characterized in that: include: A clustering processing module is configured to obtain a plurality of error logs and perform clustering processing on the plurality of error logs by a clustering algorithm to obtain a plurality of error log classes; A filtering processing module, configured to filter each of the error log classes to obtain a target error log class; A group processing module, configured to perform group processing on each of the error logs in the target error log class to obtain a plurality of first log groups; The analysis module is configured to generate an error log template corresponding to each of the first log groups based on the error logs in each of the first log groups, and analyze the cause of the failure according to the error log template.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the error log processing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the error log processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the error log processing method according to any one of claims 1 to 6 is implemented.

Images