Log processing method and device, electronic equipment and computer readable storage medium

Abstract

The application discloses a log processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of computers, which comprises the following steps: obtaining logs of a server before and after executing a test case, and then comparing to obtain logs generated in the process of executing the test case; matching the logs generated in the process of executing the test case with a rule library to obtain first target logs and second target logs; the first target logs are logs matched with rules of the rule library and having matched log analysis results, and the second target logs are logs not matched with rules of the rule library and not having matched log analysis results; and associating the first target logs with the matched log analysis results. The method can restore the test scene when a BUG problem is reproduced, improve analysis efficiency and reduce analysis omissions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a log processing method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] In the field of computer technology, log is an important tool for recording various information in the running process of the system. It is usually stored in the form of text file or database, recording the events, operations, errors, warnings and other information occurred in the running process of the system. By analyzing the log, troubleshooting, performance monitoring, security audit, system behavior analysis and other functions can be performed. However, the related technical solutions cannot restore the test site when the BUG problem is reproduced, and the analysis of the log is mostly manual, which is low in efficiency and easy to miss. Therefore, how to solve the above technical defects has become a technical problem to be solved by the technical personnel in the field. SUMMARY

[0003] The present application provides a log processing method and device, electronic equipment and computer readable storage medium, to at least solve the problem that the test site cannot be restored when the BUG problem is reproduced in the related art, and the analysis efficiency is low and easy to miss.

[0004] The present application provides a log processing method, comprising:

[0005] obtaining the log of the server before executing the test case to obtain a first log;

[0006] obtaining the log of the server after executing the test case to obtain a second log;

[0007] comparing the first log with the second log to obtain the log generated in the process of executing the test case;

[0008] matching the log generated in the process of executing the test case with a rule library to obtain a first target log and a second target log; the first target log is the log matched with the rule of the rule library and having a matched log analysis result, and the second target log is the log not matched with the rule of the rule library and not having a matched log analysis result;

[0009] associating the first target log with the matched log analysis result.

[0010] The present application also provides a log processing device, comprising:

[0011] a first obtaining module for obtaining the log of the server before executing the test case to obtain a first log;

[0012] a second obtaining module for obtaining the log of the server after executing the test case to obtain a second log;

[0013] a comparison module configured to compare the first log with the second log to obtain a log generated during execution of the test case;

[0014] a matching module configured to match the log generated during execution of the test case with a rule library to obtain a first target log and a second target log; the first target log is a log that matches a rule of the rule library and has a matched log analysis result, and the second target log is a log that does not match the rule of the rule library and does not have a matched log analysis result;

[0015] an association module configured to associate the first target log with the matched log analysis result.

[0016] The application further provides an electronic device, comprising a memory configured to store a computer program and a processor configured to execute the computer program to implement the steps of any of the log processing methods.

[0017] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of any of the log processing methods.

[0018] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps of any of the log processing methods.

[0019] The log processing method provided by the application does not clear the log of the server before execution of the test case, but adopts a manner of taking a differential log of the log of the server before and after execution of the test case to perform analysis, so that the original log can be effectively preserved from being destroyed, and the test scene can be restored when a BUG problem is reproduced subsequently. In addition, the log processing method provided by the application adopts a rule library matching manner, matches the log generated during execution of the test case with the rule library, screens out known logs and unknown logs, associates the known logs with log analysis results, can effectively improve the efficiency of log analysis, reduce manual input, improve the accuracy of analysis, and reduce analysis omissions. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 A flowchart of a log processing method provided by an embodiment of the application;

[0022] Figure 2 A specific log processing method provided for an embodiment of the present application is shown in the following figure:

[0023] Figure 3 A log and rule base comparison diagram provided for an embodiment of the present application is shown in the following figure:

[0024] Figure 4 An unknown log processing diagram provided for an embodiment of the present application is shown in the following figure:

[0025] Figure 5 A log processing device diagram provided for an embodiment of the present application is shown in the following figure:

[0026] Figure 6 An electronic device diagram provided for an embodiment of the present application is shown in the following figure. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, any other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0028] It should be noted that, in the description of the present application, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0029] In order to make the skilled in the art better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0030] Embodiments of the present application provide a log processing method, which is described in detail in combination with the execution flow of the log processing method.

[0031] Please refer to Figure 1 The log processing method provided by the embodiments of the present application includes:

[0032] S101: obtaining the log of the server before executing the test case, to obtain a first log.

[0033] S102: obtaining the log of the server after executing the test case, to obtain a second log.

[0034] S103: comparing the first log with the second log to obtain a log generated in the process of executing the test case.

[0035] The first log refers to a log of the server collected before the server executes the test case. The second log refers to a log of the server collected after the server executes the test case. The full logs of the server are collected and stored before and after the server executes the test case. The full logs include BMC (Baseboard Management Controller) logs and OS (Operating System) logs. The difference between the logs of the server collected before and after the server executes the test case is obtained to obtain a log generated in the process of executing the test case. In this embodiment, the logs of the server before and after the test case are not cleared, but the difference between the logs of the server before and after the test case is used for analysis, so that the original logs can be effectively preserved and not damaged, and the test scene can be restored when the BUG problem is reproduced.

[0036] S104: matching the log generated in the process of executing the test case with the rule library to obtain a first target log and a second target log; the first target log is a log matched with the rules of the rule library and having a matched log analysis result, and the second target log is a log not matched with the rules of the rule library and not having a matched log analysis result.

[0037] S105: associating the first target log with the matched log analysis result.

[0038] The log generated in the process of executing the test case is matched with the rule library to screen out the first target log and the second target log. The first target log, i.e., the known log, refers to a log matched with the rules of the rule library and having a corresponding matched log analysis result (including the reason for the log appearing and the solution, etc.). After the first target log is screened out, the first target log is associated with the corresponding matched log analysis result. The second target log, i.e., the unknown log, refers to a log not matched with the rules of the rule library and not having a matched log analysis result. The second target log can obtain a log analysis result through manual analysis. In this embodiment, the log generated in the process of executing the test case is matched with the rule library to screen out the known log and the unknown log, and the known log is associated with the log analysis result, which can effectively improve the efficiency of log analysis, reduce manual input, improve the accuracy of analysis, and reduce analysis omissions.

[0039] The log processing method provided by the embodiment can be implemented by a log analysis system. The log analysis system comprises a log collection module, a rule library comparison and analysis module, a log storage module, and a log re-entry module. Referring to Figure 2 As shown in the figure, before the server executes the test case, the log collection module collects full logs of the server and stores them in the log storage module. After the server executes the test case, the log collection module collects full logs of the server again and stores them in the log storage module. The log analysis comparison module takes the difference between the two full logs before and after the test to obtain logs generated during the test. The log analysis comparison module matches the logs generated during the test with the rule library maintained by the system to obtain analyzed logs and unanalyzed logs. The analyzed logs are automatically associated with the log analysis results. The unanalyzed logs are analyzed by the test engineer and manually associated with the log analysis results. The unanalyzed logs can be extracted by the log re-entry module to obtain corresponding rules, and then maintained in the rule library of the log analysis comparison module, so that the rule library is continuously expanded and enriched.

[0040] Matching the logs generated during the execution of the test case with the rule library comprises:

[0041] The logs generated during the execution of the test case are matched with the rule libraries at different levels in stages, and the logs remaining after matching with the rule library at a previous stage are matched with the rule library at a next stage.

[0042] The rule library can include a blacklist library, a whitelist library, etc. By performing log comparison and filtering in stages, fine comparison and analysis of logs can be achieved, and accuracy can be improved.

[0043] In some embodiments, matching the logs generated during the execution of the test case with the rule libraries at different levels in stages comprises:

[0044] Matching the logs generated during the execution of the test case with the blacklist library, retaining the logs matched with the blacklist library, and determining the logs not matched with the blacklist library as traffic logs and ignoring them;

[0045] Matching the logs matched with the blacklist library with the whitelist library, retaining the logs not matched with the whitelist library, and determining the logs matched with the whitelist library as traffic logs and ignoring them;

[0046] Matching the logs not matched with the whitelist library with the ignorable problem library, retaining the logs not matched with the ignorable problem library, and determining the logs matched with the ignorable problem library as traffic logs and ignoring them;

[0047] The logs that do not match the negligible problem library are matched with the common problem library, and the logs that match the common problem library are reserved; the logs that do not match the common problem library are reserved, to obtain the second target log;

[0048] The logs that match the common problem library are matched with the common negligible problem library, and the logs that do not match the common negligible problem library are reserved, to obtain the first target log, and the logs that match the common negligible problem library are determined to be traffic logs and are ignored.

[0049] In this embodiment, the log comparison is implemented based on a multi-level rule library, and the rule library mainly includes a blacklist library, a whitelist library, a negligible problem library, a common problem library, and a common negligible problem library. The blacklist library stores log rules that are known to be irrelevant or low priority (for example, debug logs, traffic logs). The whitelist library stores log rules of expected normal behavior (for example, successful connection, normal operation). The negligible problem library stores log rules that are known to be harmless but frequently occur (for example, non-critical alarms). The common problem library stores high-frequency error log rules and their solutions (for example, system crash, network interruption). The common negligible problem library stores error logs that contain error keywords and can be ignored.

[0050] The fine matching of the differential logs with the rule libraries at each level can improve accuracy.

[0051] Reference Figure 3 As shown in the figure, the log comparison and analysis process includes:

[0052] The logs generated during the test are compared with the blacklist library, and the matched logs are reserved, and the unmatched logs are determined to be traffic logs and are ignored.

[0053] The logs reserved in the previous step are compared with the whitelist library, and the matched logs are determined to be white list ignored, and the unmatched logs are reserved.

[0054] The logs reserved in the previous step are compared with the negligible problem library, and the matched logs are determined to be traffic logs and are ignored, and the unmatched logs are reserved.

[0055] The logs reserved in the previous step are compared with the common problem library, and the matched logs are determined to be common problem logs and are reserved, and the unmatched logs are determined to be unknown errors, that is, the second target logs, and are reserved, and the unknown errors are displayed to the test engineers for manual analysis.

[0056] The common problem logs reserved in the previous step are compared with the common negligible problem library, and the matched logs are determined to be traffic logs and are ignored, and the unmatched logs are reserved as known errors, that is, the first target logs, and the log analysis results are automatically associated.

[0057] Reference Figure 4As shown, after the unknown error is analyzed by the test engineer, two operations can be performed; one operation is to manually associate the log analysis result and backfill it into the use case analysis result. For example, the log analysis result is fail, and the backfilled use case analysis result is fail. One operation is to extract the rule and perfect it to the rule library at each level.

[0058] The test engineer manually analyzes the unmatched log, confirms its category and solution. The keywords, regular expressions or templates are extracted from the manual analysis result. The new rule is updated to the corresponding rule library (such as blacklist, common problem library).

[0059] In some embodiments, matching the log generated in the process of executing the test case with the rule library comprises:

[0060] Performing keyword matching on the log generated in the process of executing the test case;

[0061] Converting the log generated in the process of executing the test case into a template;

[0062] Calculating the similarity between the converted template and the templates in the rule library;

[0063] Determining whether the similarity exceeds a threshold value;

[0064] If the similarity exceeds the threshold value, it is determined that the log generated in the process of executing the test case matches the log in the rule library.

[0065] Each rule in the rule library includes keywords, regular expressions and templates. The log must contain keywords or phrases (such as "ERROR"). Regular expressions are used to match complex log patterns (such as "Received block \d+ from \d+"). Replace the variables (such as IP address, timestamp) in the log with wildcards to generate a unified template.

[0066] In the keyword matching, a string matching algorithm (such as KMP algorithm) can be used to quickly determine whether the log contains the keyword.

[0067] Using the string matching algorithm to determine whether the log contains the keyword comprises: constructing a partial matching table (next array): set the pattern string as P[0...m-1], the length is m, and define the array next[0...m-1]. Wherein: next[i]=the length of the longest common prefix and suffix (i.e. P[0...k-1]=P[i-k...i-1], and k

[0068] Initialization: next[0]=0, j=0.

[0069] Recursion from i=1 to m-1 as shown in Table 1:

[0070] Table 1

[0071]

[0072] Let S[0...n-1] be the main string, length n, and P be the pattern. The matching process is as follows:

[0073] Initialize pointers i = 0 (main string position), j = 0 (pattern string position);

[0074] While i < n and j < m:

[0075] {if S[i] = P[j], i i+1, j j+1; else, if j > 0, j next[j-1], else, i i+1}

[0076] if S[i] = P[j], else, i i+1, j j+1, if j > 0, j next[j-1], i i+1.

[0077] If j = m, the match is successful, and i - m is returned.

[0078] For example, use the kmp algorithm to determine whether "Disk usage 90%" contains the keyword "Disk usage log".

[0079] Construct the next array for the pattern string "Disk usage":

[0080] Pattern string: "Disk usage" (length m = 10, including spaces);

[0081] The next array represents the length of the longest common prefix and suffix, and the construction process is shown in Table 2:

[0082] Table 2

[0083]

[0084] This pattern string has no common prefix and suffix, so all next[i] = 0.

[0085] Matching process:

[0086] Main string: "Disk usage 90%" (length n = 13).

[0087] Pattern string: "Disk usage" (length m = 10).

[0088] Matching step:

[0089] Initialize i = 0 (main string pointer), j = 0 (pattern string pointer).

[0090] Compare characters one by one:

[0091] The first 10 characters of the main string "Disk usage 90%" match the pattern string "Disk usage" completely.

[0092] When j = 10, the pattern string has been completely matched, and the matching position i - m = 0 is returned.

[0093] The log after matching the keyword is considered to contain alarm information. Next, the log in the log library is matched to determine the error type, which can be matched using template similarity calculation:

[0094] After converting the log into a template, similarity calculation is performed with the templates in the rule library.

[0095] In some embodiments, calculating the similarity between the template obtained by converting the log and the templates in the rule library includes:

[0096] Calculating the length of the longest common subsequence of the template obtained by converting the log and the templates in the rule library;

[0097] According to the length of the longest common subsequence calculated, the length of the template obtained by converting the log, and the length of the template in the rule library, the harmonic mean similarity of the template obtained by converting the log and the template in the rule library is calculated.

[0098] If the harmonic mean similarity exceeds a threshold value (e.g., 0.8), it is determined to be a successful match.

[0099] The formula for calculating the harmonic mean similarity is as follows:

[0100] .

[0101] Similarity represents the harmonic mean similarity. LCS length represents the length of the longest common subsequence, len(X) represents the length of sequence X, and len(Y) represents the length of sequence Y.

[0102] For example, X = "ABCD" and Y = "ACD".

[0103] Similarity = 2 x 3 / (4 + 3) ≈ 0.857.

[0104] LCS (Longest Common Subsequence) is used to find the longest common subsequence in two sequences (subsequence does not require consecutive, but the order must match). The following is its core calculation method and formula:

[0105] Let two sequences be X=x1x2...xn and Y=y1y2...ym, define two-dimensional array dp[i][j], which represents the LCS length of X[1..i] and Y[1..j].

[0106] The recurrence formula is as follows:

[0107] .

[0108] Initialization: when i=0 or j=0, dp[0][j]=0, and dp[i][0]=0. Because the empty sequence has no common subsequence.

[0109] Calculation steps:

[0110] 1. Construct the dynamic programming table:

[0111] From i=1 to n, j=1 to m, fill in the table according to the recurrence formula.

[0112] 2. Backtracking to find the specific subsequence:

[0113] Start from dp[n][m], and inversely trace the path:

[0114] If xi=yj, the character belongs to LCS, move to the left and up (i i-1, j j-1).

[0115] If not, move to the larger direction (left or up).

[0116] For example, assume X="ABCBDAB", Y="BDCAB", and its dynamic programming table is shown in Table 3 (part):

[0117] Table 3

[0118]

[0119] The longest common subsequence is "BCAB" or "BDAB", both with a length of 4.

[0120] This embodiment modularizes the log processing, and realizes accurate matching of the log through keyword matching and template similarity calculation, improving the accuracy.

[0121] In some embodiments, it also includes;

[0122] predicting an abnormal type of the second target log, and perfecting the rule base according to the second target log.

[0123] wherein perfecting the rule base according to the second target log comprises:

[0124] grouping the second target log to generate a new template;

[0125] extracting keywords and variables in the second target log to generate a new rule.

[0126] For unknown logs, i.e. logs that do not match existing rules, the embodiment automatically performs rule extraction, and the extracted new rules are added to the rule base. Matching with the rule base is mainly realized by keyword screening. The rule base is automatically perfected, and under the condition that the keywords are comprehensive enough, there will be no problem of analysis omission.

[0127] In some embodiments, extracting a new rule according to the second target log comprises:

[0128] predicting an abnormal type of the second target log;

[0129] grouping the second target log to generate a new template;

[0130] extracting keywords and variables in the second target log.

[0131] For logs that do not match existing rules, machine learning analysis is performed, and LSTM (Long Short-Term Memory) or BERT (Bidirectional Encoder Representations from Transformers) model can be used to predict the abnormal type of the log. For example, the log "Network latency exceeded 200ms" is classified as "network delay anomaly".

[0132] For logs that do not match existing rules, clustering and pattern mining are performed, and LogCluster or IPLoM (Iterative Partitioning Log Mining) algorithm can be used to group the second target log to generate a new template.

[0133] For example, "Disk usage 90%" and "Disk usage 85%" are clustered as "Disk usage <*>%".

[0134] For the logs that are not matched to the existing rules, dynamic rule generation is performed, keywords and variables in the logs are extracted, new rules are generated and updated to the rule library.

[0135] For example, from the log "Received block 123 from 192.168.1.1", the rule "Received block \d+ from \d+" is extracted.

[0136] In some embodiments, the stored logs of the server are associated with the project, the model and the test case.

[0137] In this embodiment, each log enters the corresponding storage library with the attached labels, including the project, the model, the test case number and the corresponding rule, so that the logs are associated with the project, the model and the test case, which can facilitate the filtering of key logs according to the project, the model and the test case, and is more convenient for analyzing the logs.

[0138] In summary, the log processing method provided in the application does not clear the logs of the server before executing the test case, but uses the differential logs of the logs of the server before and after executing the test case for analysis, which can effectively preserve the original logs from being destroyed, and can restore the test site when the BUG problem is reproduced. In addition, the log processing method provided in the application uses the rule library matching method to match the logs generated during the execution of the test case with the rule library, filters out the known logs and the unknown logs, associates the known logs with the log analysis results, which can effectively improve the efficiency of log analysis, reduce the manual input, improve the accuracy of analysis and reduce the missed problems in analysis.

[0139] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0140] The embodiments of the application also provide a log processing device, as shown in Figure 5 The log processing device comprises:

[0141] The first acquisition module 10 is configured to acquire the logs of the server before executing the test case, and obtain the first logs.

[0142] The second acquisition module 20 is configured to acquire the logs of the server after executing the test case, and obtain the second logs.

[0143] The comparison module 30 is configured to compare the first logs with the second logs, and obtain the logs generated during the execution of the test case.

[0144] The matching module 40 is configured to match the logs generated in the process of executing the test case with the rule library to obtain first target logs and second target logs; the first target logs are logs matched with the rules of the rule library and having matched log analysis results, and the second target logs are logs not matched with the rules of the rule library and not having matched log analysis results.

[0145] The association module 50 is configured to associate the first target logs with the matched log analysis results.

[0146] In the above embodiment, as a specific implementation, the matching module 40 is configured to:

[0147] match the logs generated in the process of executing the test case with the rule libraries at different levels in stages; and the logs reserved after matching with the rule library at a previous level are matched with the rule library at a next level.

[0148] In the above embodiment, as a specific implementation, the matching module 40 includes:

[0149] The first matching unit is configured to match the logs generated in the process of executing the test case with the blacklist library, retain the logs matched with the blacklist library, and determine the logs not matched with the blacklist library as traffic logs and ignore the logs.

[0150] The second matching unit is configured to match the logs matched with the blacklist library with the whitelist library, retain the logs not matched with the whitelist library, and determine the logs matched with the whitelist library as traffic logs and ignore the logs.

[0151] The third matching unit is configured to match the logs not matched with the whitelist library with the ignorable problem library, retain the logs not matched with the ignorable problem library, and determine the logs matched with the ignorable problem library as traffic logs and ignore the logs.

[0152] The fourth matching unit is configured to match the logs not matched with the ignorable problem library with the common problem library, retain the logs matched with the common problem library, and retain the logs not matched with the common problem library to obtain the second target logs.

[0153] The fifth matching unit is configured to match the logs matched with the common problem library with the common ignorable problem library, retain the logs not matched with the common ignorable problem library to obtain the first target logs, and determine the logs matched with the common ignorable problem library as traffic logs and ignore the logs.

[0154] In the above embodiment, as a specific implementation, the matching module 40 includes:

[0155] The keyword matching unit is configured to perform keyword matching on the log generated in the process of executing the test case.

[0156] The conversion unit is configured to convert the log generated in the process of executing the test case into a template.

[0157] The calculation unit is configured to calculate the similarity between the template obtained by conversion and the templates in the rule library.

[0158] The judgment unit is configured to judge whether the similarity exceeds a threshold.

[0159] The determination unit is configured to determine that the log generated in the process of executing the test case matches the log in the rule library if the similarity exceeds the threshold.

[0160] On the basis of the above-mentioned embodiments, as a specific implementation, the method further comprises:

[0161] The prediction and improvement module is configured to predict the abnormal type of the second target log and improve the rule library according to the second target log.

[0162] On the basis of the above-mentioned embodiments, as a specific implementation, the prediction and improvement module comprises:

[0163] The grouping unit is configured to group the second target log to generate a new template.

[0164] The extraction unit is configured to extract keywords and variables in the second target log to generate a new rule.

[0165] On the basis of the above-mentioned embodiments, as a specific implementation, the stored log of the server is associated with a project, a model and a test case.

[0166] The features of the embodiments of the log processing device can be understood with reference to the related descriptions of the embodiments of the log processing method, which will not be repeated here.

[0167] Embodiments of the present application also provide an electronic device, as shown in Figure 6 The electronic device comprises a memory 1 and a processor 2, the memory 1 stores a computer program, and the processor 2 is configured to run the computer program to execute the steps in any of the above-mentioned log processing method embodiments.

[0168] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above-mentioned log processing method embodiments when running.

[0169] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0170] Embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program, when executed by a processor, implements the steps in any of the log processing method embodiments described above.

[0171] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps in any of the log processing method embodiments described above.

[0172] The skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0173] The above describes in detail a log processing method, device, electronic equipment and computer readable storage medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples in this paper, and the above description of the examples is only used to help understand the method and its core idea of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the present application.

Claims

1. A log processing method, characterized in that: include: Get the server log before executing the test case and get the first log; Obtaining a log of the server after executing the test case to obtain a second log; The server logs are not cleared before and after executing the test cases; Comparing the first log with the second log to obtain a log generated during the execution of the test case; Matching the logs generated during the execution of the test case with the rule base to obtain a first target log and a second target log; the first target log is a log that matches the rules of the rule base and has a matching log analysis result, and the second target log is a log that does not match the rules of the rule base and does not have a matching log analysis result; the log analysis result includes the cause of the log and a solution; Associating the first target log with the matching log analysis result; Matching the logs generated during the test case execution with the rule base includes: Match the logs generated during the execution of the test case with the blacklist library, retain the logs that match the blacklist library, and determine that the logs that do not match the blacklist library are traffic logs and ignore them; Match the logs that match the blacklist library with the whitelist library, retain the logs that do not match the whitelist library, and determine that the logs that match the whitelist library are traffic logs and ignore them; Matching the logs that do not match the whitelist library with the ignorable problem library, retaining the logs that do not match the ignorable problem library, and determining the logs that match the ignorable problem library as traffic logs and ignoring them; Matching the logs that do not match the ignorable question library with the common question library, retaining the logs that match the common question library; retaining the logs that do not match the common question library, and obtaining the second target logs; Match the logs that match the common problem library with the common ignorable problem library, retain the logs that do not match the common ignorable problem library, obtain the first target log, and determine that the logs that match the common ignorable problem library are traffic logs and ignore them.

2. The log processing method according to claim 1, characterized in that: Matching the logs generated during the test case execution with the rule base includes: Perform keyword matching on the logs generated during the execution of test cases; Convert the logs generated during the execution of test cases into templates; Calculating the similarity between the converted template and the template in the rule base; Determining whether the similarity exceeds a threshold; If the similarity exceeds a threshold, it is determined that the log generated during the execution of the test case matches the log in the rule base.

3. The log processing method according to claim 1, wherein: Also includes: Predict the abnormal type of the second target log, and improve the rule base according to the second target log.

4. The log processing method according to claim 3, characterized in that: Improving the rule base according to the second target log includes: Group the second target logs and generate a new template; Extract keywords and variables from the second target log to generate new rules.

5. The log processing method according to claim 1, wherein: The stored logs of the server are associated with projects, models, and test cases.

6. A log processing device, characterized in that: include: A first acquisition module is used to obtain the log of the server before executing the test case to obtain a first log; A second acquisition module is used to obtain the log of the server after executing the test case to obtain a second log; The server logs are not cleared before and after executing the test cases; A comparison module, configured to compare the first log with the second log to obtain a log generated during the execution of the test case; A matching module is configured to match the logs generated during the execution of the test case with the rule base to obtain a first target log and a second target log; the first target log is a log that matches the rules of the rule base and has a matching log analysis result, and the second target log is a log that does not match the rules of the rule base and does not have a matching log analysis result; the log analysis result includes the cause of the log and a solution; an associating module, configured to associate the first target log with a matching log analysis result; The matching modules include: a first matching unit, configured to match logs generated during the execution of a test case with a blacklist library, retain logs that match the blacklist library, and determine that logs that do not match the blacklist library are traffic logs and ignore them; a second matching unit, configured to match the logs that match the blacklist library with the whitelist library, retain the logs that do not match the whitelist library, and determine that the logs that match the whitelist library are traffic logs and ignore them; a third matching unit, configured to match logs that do not match the whitelist library with an ignorable problem library, retain logs that do not match the ignorable problem library, and determine that logs that match the ignorable problem library are traffic logs and ignore them; a fourth matching unit, configured to match the logs that do not match the ignorable question library with the common question library, retain the logs that match the common question library; retain the logs that do not match the common question library, and obtain the second target logs; The fifth matching unit is used to match the logs that match the common problem library with the common ignorable problem library, retain the logs that do not match the common ignorable problem library, obtain the first target log, and determine that the logs that match the common ignorable problem library are traffic logs and ignore them.

7. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the log processing method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the log processing method according to any one of claims 1 to 5 are implemented.

Images