Error log analysis method and apparatus, and storage medium and computer device

Abstract

An error log analysis method and apparatus, and a storage medium and a computer device, which relate to the technical field of computers. A text similarity between a first error log recorded within the latest recording time period and a second error log recorded within a historical recording time period can be calculated using a text similarity algorithm, and whether similar text content is recorded in the first error log and the second error log is determined accordingly, so as to realize log comparison, such that whether an error log of a new error type is recorded within the latest recording time period can be determined. Thus, accurate log analysis can be performed on an error log in an unknown format, without the need for pre-configuring a log format, thereby improving the accuracy of a log analysis result. In addition, the manual involvement degree and the manual workload can also be greatly reduced, thereby improving the log comparison analysis efficiency.

Description

Error log analysis method, device, storage medium and computer equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 22, 2023, with application number 202311791409.5 and invention name “Error log analysis method, device, storage medium and computer equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to an error log analysis method, apparatus, storage medium, and computer equipment. Background Art

[0003] System logs are data used to record hardware, software, and / or error information during system operation. Analyzing error logs, which record these errors, can help developers understand system status, identify and diagnose issues, and optimize system performance. Due to the sheer volume of error log data, manual log analysis is both labor-intensive and time-consuming, and can also result in biased results. Therefore, existing technologies have proposed automated log analysis using computer equipment.

[0004] Currently, existing automated log analysis solutions focus on structured log analysis. This involves extracting key information from error logs based on a pre-defined log format and generating analysis results based on this information. However, in real-world applications, log formats vary, making it difficult for developers to pre-configure all log formats into computer devices. Existing solutions are unable to analyze error logs for which no pre-configured log format exists, reducing the accuracy of log analysis results.

[0005] Summary of the Invention

[0006] The purpose of this application is to solve at least one of the above technical deficiencies, especially the technical defect of low accuracy of log analysis results in the prior art.

[0007] In a first aspect, an embodiment of the present application provides an error log analysis method, the method comprising:

[0008] When the current timing moment arrives, query the first error log recorded in the latest recording period; the end time of the latest recording period is the current timing moment;

[0009] If at least one first error log is recorded in the latest recording period, querying the second error log recorded in the historical recording period; the end time of the historical recording period is not later than the start time of the latest recording period;

[0010] If at least one second error log is recorded within the historical record period, performing text processing on each of the first error logs to obtain at least one first log text, and performing text processing on each of the second error logs to obtain at least one second log text;

[0011] For each of the first log texts, respectively calculating a first text similarity between the first log text and each of the second log texts, and determining whether the first log text is similar to at least one of the second log texts based on each of the first text similarities, and obtaining a similarity determination result;

[0012] A first log analysis result is generated based on each of the similarity judgment results, where the first log analysis result is used to reflect whether an error log of a newly added error type is recorded in the latest recording period.

[0013] In one embodiment, generating a first log analysis result based on each of the similarity determination results includes:

[0014] According to each of the similarity judgment results, it is judged whether there is an abnormal log text; wherein the abnormal log text is a first log text that is not similar to each of the second log texts;

[0015] If there is at least one of the abnormal log texts, each of the abnormal log texts is used as a query keyword to perform a log query on the third error log recorded in the verification record period, and obtain a first query result corresponding to each of the abnormal log texts; wherein the end time of the verification record period is not earlier than the start time of the historical record period, and the duration of the verification record period is greater than the duration of the historical record period;

[0016] The first log analysis result is generated according to each of the first query results.

[0017] In one embodiment, the performing text processing on each of the first error logs to obtain at least one first log text, and the performing text processing on each of the second error logs to obtain at least one second log text, include:

[0018] Performing text preprocessing on each of the first error logs to obtain a first preprocessed text corresponding to each of the first error logs;

[0019] Performing text preprocessing on each of the second error logs to obtain a second preprocessed text corresponding to each of the second error logs;

[0020] Calculating a second text similarity between each pair of the first preprocessed texts, and calculating a third text similarity between each pair of the second preprocessed texts;

[0021] Deduplication is performed on each of the first preprocessed texts according to the similarity of each of the second texts to obtain at least one of the first log texts;

[0022] Deduplication is performed on each of the second preprocessed texts according to each of the third text similarities to obtain at least one of the second log texts.

[0023] In one embodiment, the method further comprises:

[0024] If at least one first error log is recorded in the latest recording period, calculating the difference between the number of logs in the first error log and the number of logs in the second error log;

[0025] A second log analysis result is generated based on a magnitude relationship between the quantity difference and a preset quantity threshold, and is used to reflect whether there is a sudden increase in error logs in the latest recording period.

[0026] In one embodiment, the method further comprises:

[0027] If no second error log is recorded within the historical record period, text processing is performed on each of the first error logs to obtain at least one first log text, and each of the first log texts is used as a query keyword to perform a log query on the third error log recorded within the verification record period, and obtain a second query result corresponding to each first log text; wherein the end time of the verification record period is no earlier than the start time of the historical record period, and the duration of the verification record period is greater than the duration of the historical record period;

[0028] The first log analysis result is generated according to each of the second query results.

[0029] In one embodiment, querying the first error log recorded in the latest recording period includes:

[0030] Calling a log quantity query interface according to the latest record period to obtain the log quantity of the first error log returned by the log quantity query interface;

[0031] If the number of logs in the first error log is greater than or equal to zero, a log text query interface is called according to the latest record period to obtain each of the first error logs returned by the log text query interface.

[0032] In one embodiment, the method further comprises:

[0033] If the number of logs in the first error log is zero, a first log analysis result is generated to reflect that no error log of a new error type is recorded in the latest recording period.

[0034] In a second aspect, an embodiment of the present application provides an error log analysis device, the device comprising:

[0035] A first query module is configured to query the first error log recorded in the latest recording period when the current timing moment arrives; the end time of the latest recording period is the current timing moment;

[0036] a second query module configured to query a second error log recorded in a historical record period if at least one first error log is recorded in the latest record period; the end time of the historical record period being no later than the start time of the latest record period;

[0037] a text processing module, configured to, if at least one second error log is recorded within the historical record period, perform text processing on each of the first error logs to obtain at least one first log text, and perform text processing on each of the second error logs to obtain at least one second log text;

[0038] a log similarity determination module, configured to calculate, for each of the first log texts, a first text similarity between the first log text and each of the second log texts, and determine, based on each first text similarity, whether the first log text is similar to at least one of the second log texts, and obtain a similarity determination result;

[0039] The first result generating module is configured to generate a first log analysis result based on each of the similarity judgment results, wherein the first log analysis result is configured to reflect whether an error log of a new error type is recorded in the latest recording period.

[0040] In a third aspect, an embodiment of the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the error log analysis method described in any of the above embodiments.

[0041] In a fourth aspect, an embodiment of the present application provides a computer device, the computer device comprising: one or more processors, and a memory;

[0042] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the error log analysis method described in any of the above embodiments are executed.

[0043] In the error log analysis method, apparatus, storage medium, and computer equipment provided in some embodiments of the present application, a text similarity algorithm can be used to calculate the text similarity between a first error log recorded in the latest recording period and a second error log recorded in the historical recording period, and based on this, it can be determined whether similar text content is recorded between the first error log and the second error log, thereby achieving log comparison, and then determining whether an error log of a newly added error type is recorded in the latest recording period. In this way, there is no need to pre-configure the log format, and accurate log analysis can be performed on error logs in unknown formats, thereby improving the accuracy of the log analysis results. In addition, it can also greatly reduce manual participation and manual workload, thereby improving the efficiency of log comparison analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] FIG1 is a schematic diagram of a flow chart of an error log analysis method according to an embodiment;

[0046] FIG2 is a schematic diagram of the structure of an error log analysis device according to an embodiment;

[0047] FIG3 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] In one embodiment, the present application provides an error log analysis method. The following embodiments illustrate this method using a computer device as an example. It should be understood that the computer device described herein refers to a device with data acquisition and data processing capabilities, and may include, but is not limited to, various servers, personal computers, and laptop computers.

[0050] As shown in Figure 1, the error log analysis method provided by this application may specifically include the following steps:

[0051] S102: When the current timing moment arrives, query the first error log recorded within the latest recording period.

[0052] Among them, the end moment of the latest recording period can be the current timing moment, and the start moment of the latest recording period can be determined according to actual situations such as the preset timing duration, log comparison frequency, and / or log comparison range, etc., and this text does not make specific restrictions on this. The first error log refers to the error log recorded within the latest recording period.

[0053] It should be noted that the "within..." described in this text can be understood as not including the endpoint value, including one side endpoint value, or including both side endpoint values, and this text does not make specific restrictions on this. Exemplarily, if the start moment of the latest recording period is Ts, the end moment is Te, the middle moment is Tm, Ts < Tm < Te, and there is an error log Ls recorded at the moment Ts, an error log Lm recorded at the moment Tm, and an error log Le recorded at the moment Te, then the first error log recorded within the latest period can be one of the following four situations:

[0054] (1) The first error log includes Lm, and does not include Ls and Le;

[0055] (2) The first error log includes Ls and Lm, and does not include Le;

[0056] (3) The first error log includes Lm and Le, and does not include Ls; [[ID=第十六]] [[ID=第十七]]

[0057] (4) The first error log includes Ls, Lm and Le.

[0058] In this step, the computer device can query the first error log recorded within the latest recording period through a timing task. It can be understood that the timing duration of the timing task can be determined according to actual situations, and this text does not make specific restrictions on this. In one example, the timing duration can be 2 minutes, for example, query the error log recorded within 2 minutes every 2 minutes.

[0059] S104: If there is at least one first error log recorded within the latest recording period, query the second error log recorded within the historical recording period.

[0060] Among them, the end moment of the historical recording period is earlier than or equal to the start moment of the latest recording period, and the start moment of the historical recording period can be determined according to actual situations such as the preset timing duration, log comparison frequency, and / or log comparison range, etc., and this text does not make specific restrictions on this. Further, in one example, the duration of the historical recording period can be equal to the duration of the latest recording period, and the end moment of the historical recording period is the start moment of the latest recording period.

[0061] The second error log refers to the error log recorded during the historical record period. For relevant instructions on the second error log, please refer to the description of the first error log above, and this article will not repeat them here.

[0062] In this step, if at least one first error log is obtained through the query, it indicates that an error log is recorded in the latest recording period. Therefore, the second error log recorded in the historical recording period can be queried to facilitate subsequent comparative analysis of the first error log based on the second error log.

[0063] S106: If at least one second error log is recorded in the history recording period, text processing is performed on each first error log to obtain at least one first log text, and text processing is performed on each second error log to obtain at least one second log text.

[0064] In this step, if at least one second error log is obtained through the query, the first error log and the second error log need to be compared to determine whether each first error log has similar text content to the at least one second error log.

[0065] Specifically, the computer device may perform text processing on each first error log and obtain at least one first log text. Each first log text obtained by the computer device may reflect the text content of each first error log. Similar to the processing of the first error log, the computer device may perform text processing on each second error log to obtain at least one second log text. Each second log text may reflect the text content of each second error log.

[0066] It is understood that the specific method of text processing can be determined based on actual conditions such as computing resource requirements and accuracy requirements, and this document does not impose specific restrictions on this. For example, the text processing of this application may include at least one or any combination of processing methods such as text vector calculation, text cleaning, and word segmentation.

[0067] S108: For each first log text, respectively calculate the first text similarity between the first log text and each second log text, and determine whether the first log text is similar to at least one second log text according to each first text similarity, and obtain a similarity determination result.

[0068] It is understood that any text similarity algorithm can be used herein to calculate the first text similarity. For example, Euclidean distance, cosine similarity, or Pearson correlation coefficient can be used as the first text similarity, and this application does not impose any specific restrictions on this.

[0069] Since the first text similarity can reflect the degree of similarity in text content between the first log text used to calculate the first text similarity and the second log text, the computer device can determine whether the first log text and the second log text have similar text content based on the first text similarity, and thus obtain a similarity determination result corresponding to each first log text. The similarity determination result is used to reflect whether the corresponding first log text is similar to at least one second log text.

[0070] For example, when the first log text includes a first text and a second text, and the second log text includes a third text and a fourth text, the computer device can respectively calculate the first text similarity between the first text and the third text, the first text similarity between the first text and the fourth text, the first text similarity between the second text and the third text, and the first text similarity between the second text and the fourth text, and obtain the similarity judgment result corresponding to the first text and the similarity judgment result corresponding to the second text through the aforementioned four first text similarities.

[0071] S110: Generate a first log analysis result based on each similarity determination result.

[0072] The first log analysis result is used to reflect whether an error log of a new error type is recorded in the latest recording period, that is, whether at least one first error log is an error log recorded for the first time.

[0073] In this application, a text similarity algorithm can be used to calculate the text similarity between the first error log recorded in the latest recording period and the second error log recorded in the historical recording period, and based on this, it can be determined whether the first error log and the second error log record similar text content, to achieve log comparison, and then it can be determined whether the latest recording period records error logs of new error types. In this way, there is no need to pre-configure the log format, and accurate log analysis can be performed on error logs in unknown formats, thereby improving the accuracy of the log analysis results. In addition, it can also greatly reduce manual participation and manual workload, thereby improving the efficiency of log comparison and analysis.

[0074] In one embodiment, generating a first log analysis result based on each similarity determination result includes:

[0075] Step A1: judging whether there is an abnormal log text based on each similarity judgment result; wherein the abnormal log text is a first log text that is not similar to each second log text;

[0076] Step A3: If at least one abnormal log text exists, each abnormal log text is used as a query keyword to perform a log query on the third error log recorded during the verification record period, and obtain a first query result corresponding to each abnormal log text; wherein the end time of the verification record period is not earlier than the start time of the historical record period, and the duration of the verification record period is longer than the duration of the historical record period;

[0077] Step A5: Generate a first log analysis result according to each first query result.

[0078] The third error log refers to the error log recorded during the verification period. For details about the third error log, please refer to the description of the first error log above and will not be repeated here. The verification period can be a period used to repeatedly verify the exception log text. Its duration is longer than the historical record period and its end time is no earlier than the start time of the historical record period. In one example, the verification period can last for 12 hours and end at the current time.

[0079] Specifically, the computer device can use each similarity determination result to preliminarily screen out abnormal log texts from each first log text that are not similar to the text of each second log text. For example, if the first text similarity between the first log text and each second log text is less than or equal to a preset similarity threshold, it can be determined that the text content of the first log text is not similar to the text content of each second log text, and the first log text is an abnormal log text.

[0080] If there is at least one abnormal log text in each first log text, a log query can be initiated using each abnormal log text as a query keyword to query whether each abnormal log text is recorded in the verification record period, and obtain the first query result corresponding to each abnormal log text. For example, when including a first abnormal log text and a second abnormal log text, the computer device can use the first abnormal log text as a query keyword to query the third error log in the verification record period, and obtain the first query result corresponding to the first abnormal log text. The computer device can also use the second abnormal log text as a query keyword to query the third error log in the verification record period, and obtain the first query result corresponding to the second abnormal log.

[0081] The first query result corresponding to each abnormal log text can be used to reflect whether a third error log similar to the abnormal log text is recorded during the verification record period. The computer device can determine whether an error log of a new error type appears in the most recent record period based on each first query result and generate a first log analysis result.

[0082] In one example, for each exception log text, if the first query result corresponding to the exception log text reflects that at least one third error log is queried within the verification record period, then it indicates that the first error log corresponding to the exception log text has been recorded within the verification record period, and the first error log is not a newly added error type. If the first query result corresponding to the exception log text reflects that no third error log is queried within the verification record period, then it indicates that the first error log corresponding to the exception log text is not recorded within the verification record period, and the first error log is a newly added error type.

[0083] In this embodiment, abnormal log texts are preliminarily screened out based on a shorter historical record period, and abnormal repeated verification is performed on the abnormal log texts using a longer verification record period. This can reduce the consumption of computing resources on the one hand, and improve the accuracy of log analysis on the other hand.

[0084] In one embodiment, performing text processing on each first error log to obtain at least one first log text, and performing text processing on each second error log to obtain at least one second log text, includes:

[0085] Step B1: performing text preprocessing on each first error log to obtain a first preprocessed text corresponding to each first error log;

[0086] Step B3: performing text preprocessing on each second error log to obtain a second preprocessed text corresponding to each second error log;

[0087] Step B5: calculating the second text similarity between each pair of the first preprocessed texts, and calculating the third text similarity between each pair of the second preprocessed texts;

[0088] Step B7: performing text deduplication on each first pre-processed text according to the similarity of each second text to obtain at least one first log text;

[0089] Step B9: Deduplication is performed on each second pre-processed text according to each third text similarity to obtain at least one second log text.

[0090] In this embodiment, in the process of obtaining the first log text, the computer device may perform text preprocessing on each first error log respectively to obtain the first preprocessed text corresponding to each first error log. The computer device may calculate the second text similarity between each first preprocessed text, and deduplicate each first preprocessed text based on the calculated second text similarity, so as to represent multiple first preprocessed texts with similar text content through one first log text. In the first log text obtained after deduplication, if the number of first log texts is greater than or equal to 2, then the texts between each first log text are not similar.

[0091] It is understood that the text preprocessing method / algorithm used in this application can be determined based on actual circumstances and may include, but is not limited to, at least one of data cleaning, word segmentation, stop word removal, word segmentation, and word duplication removal. In this way, key information in the first error log can be automatically extracted and the corresponding first preprocessed text can be obtained.

[0092] Exemplarily, in the process of deduplicating each first preprocessed text, if the first preprocessed texts are the first text, the second text, and the third text respectively, the computer device may respectively calculate the second text similarity between the first text and the second text, the second text similarity between the first text and the third text, and the second text similarity between the second text and the third text. When the second text similarity is greater than the preset similarity threshold, it indicates that the two first preprocessed texts for calculating the second text similarity have similar text content. The computer device may retain one of the first preprocessed texts as the first log text, and not use the other first preprocessed text as the first log text.

[0093] Similar to the process of obtaining the first log text, in the process of obtaining the second log text, the computer device can perform text preprocessing on each second error log respectively to obtain the second preprocessed text corresponding to each second error log. The computer device can calculate the third text similarity between each second preprocessed text and deduplicate each second preprocessed text based on the calculated third text similarity to obtain at least one second log text. For the specific process of obtaining the second log text, please refer to the relevant description of obtaining the first log text above, and this article will not repeat it here.

[0094] In this embodiment, by deduplicating the first error log to obtain the first log text, and deduplicating the second error log to obtain the second log text, and performing subsequent log analysis based on the first log text and the second log text, the data processing volume can be reduced to further reduce the consumption of computing resources.

[0095] In one embodiment, the error log analysis method of the present application may further include the following steps:

[0096] If at least one first error log is recorded in the latest recording period, the difference between the number of logs in the first error log and the number of logs in the second error log is calculated;

[0097] According to the size relationship between the quantity difference and the preset quantity threshold, a second log analysis result is generated to reflect whether there is a sudden increase in error logs in the latest recording period.

[0098] In this embodiment, the computer device can calculate the difference between the number of first error logs recorded in the most recent recording period and the number of second error logs recorded in the historical recording period, thereby reflecting the change in the number of error logs between the most recent recording period and the historical recording period through the difference in number. The computer device can compare the difference in number with a preset threshold and, based on the comparison result, determine whether there is an abnormal sudden increase in the number of error logs in the most recent recording period. In this way, the computer device can automatically identify sudden increases in error logs, further reducing manual intervention and improving log analysis efficiency.

[0099] In one example, if the quantity difference is greater than or equal to a preset quantity threshold, the computer device may determine that there is a sudden increase in error logs in the latest recording period; otherwise, the computer device may determine that there is no sudden increase in error logs in the latest recording period.

[0100] In one embodiment, the error log analysis method of the present application may further include the following steps:

[0101] If no second error log is recorded in the historical record period, text processing is performed on each first error log to obtain at least one first log text, and each first log text is used as a query keyword to perform a log query on the third error log recorded in the verification record period, and a second query result corresponding to each first log text is obtained; wherein the end time of the verification record period is not earlier than the start time of the historical record period, and the duration of the verification record period is greater than the duration of the historical record period;

[0102] A first log analysis result is generated according to each second query result.

[0103] In this embodiment, if no second error log is recorded during the historical record period, it indicates that each first error log is a new error type relative to the historical record period, and each first error log needs to be subjected to abnormal repeated verification. In this case, the computer device can perform text processing on each first error log to obtain at least one first log text. The computer device can initiate a log query using each first log text as a query keyword to query whether the corresponding first log text is recorded during the verification record period, and obtain the second query result corresponding to each first log text. The specific acquisition process of the first log text can refer to the above embodiment, and the specific acquisition process of the second query result can refer to the acquisition process of the above-mentioned first query result, which will not be repeated here.

[0104] The second query result corresponding to each first log text can be used to reflect whether a third error log similar to the first log text is recorded during the verification record period. Therefore, the computer device can determine whether an error log of a new error type appears in the recent record period based on each second query result and generate a first log analysis result.

[0105] This embodiment uses a shorter historical record period for preliminary screening and a longer verification record period for abnormal repetition verification, which can reduce the consumption of computing resources on the one hand and improve the accuracy of log analysis on the other hand.

[0106] In one embodiment, querying the first error log recorded in the latest recording period includes:

[0107] Calling the log quantity query interface according to the latest record period to obtain the log quantity of the first error log returned by the log quantity query interface;

[0108] If the number of logs in the first error log is greater than or equal to zero, the log text query interface is called according to the latest record period to obtain each first error log returned by the log text query interface.

[0109] In this embodiment, the computer device can call the log quantity query interface to query whether there are any first error logs recorded in the most recent recording period. If so, the computer device can then call the log text query interface to query the log text information of each first error log. This can improve the reliability and accuracy of data query, further enhancing the accuracy of log analysis results.

[0110] Similarly, in one embodiment, querying the second error log recorded in the historical record period may include:

[0111] Call the log quantity query interface according to the historical record period to obtain the log quantity of the second error log returned by the log quantity query interface;

[0112] If the number of logs in the second error log is greater than or equal to zero, the log text query interface is called according to the historical record period to obtain each second error log returned by the log text query interface.

[0113] In one embodiment, the error log analysis method of the present application further includes the step of: if the number of logs in the first error log is zero, generating a first log analysis result reflecting that no error logs of a new error type are recorded in the latest recording period.

[0114] In this embodiment, if the number of first error logs returned by the log number query interface is zero, it indicates that no first error logs were recorded in the most recent recording period. Therefore, it can be directly determined that no error logs of the newly added error type were recorded in the most recent recording period. This can further improve the efficiency of log analysis.

[0115] The error log analysis device provided in an embodiment of the present application is described below. The error log analysis device described below and the error log analysis method described above can be referenced to each other.

[0116] In one embodiment, the present application provides an error log analysis device 200. As shown in FIG2 , the device 200 includes a first query module 210, a second query module 220, a text processing module 230, a log similarity determination module 240, and a first result generation module 250.

[0117] The first query module 210 is configured to query the first error log recorded in the latest recording period when the current timing moment arrives; the end time of the latest recording period is the current timing moment;

[0118] A second query module 220 is configured to query a second error log recorded in a historical record period if at least one first error log is recorded in the latest record period; the end time of the historical record period is no later than the start time of the latest record period;

[0119] a text processing module 230 configured to, if at least one second error log is recorded within the historical record period, perform text processing on each of the first error logs to obtain at least one first log text, and perform text processing on each of the second error logs to obtain at least one second log text;

[0120] The log similarity determination module 240 is configured to calculate, for each of the first log texts, a first text similarity between the first log text and each of the second log texts, and determine, based on each first text similarity, whether the first log text is similar to at least one of the second log texts, and obtain a similarity determination result;

[0121] The first result generating module 250 is configured to generate a first log analysis result based on each of the similarity judgment results, where the first log analysis result is used to reflect whether an error log of a new error type is recorded in the latest recording period.

[0122] In one embodiment, the first result generation module 250 includes an abnormal log text judgment unit, a log query unit, and a log analysis unit. The abnormal log text judgment unit is used to judge whether there is an abnormal log text based on each of the similarity judgment results; wherein the abnormal log text is a first log text that is not similar to each of the second log texts. The log query unit is used to perform a log query on the third error log recorded in the verification record period using each of the abnormal log texts as a query keyword if there is at least one abnormal log text, and obtain the first query result corresponding to each abnormal log text; wherein the end time of the verification record period is not earlier than the start time of the historical record period, and the duration of the verification record period is greater than the duration of the historical record period. The log analysis unit is used to generate the first log analysis result based on each of the first query results.

[0123] In one embodiment, the text processing module 230 includes a first preprocessing unit, a second preprocessing unit, a text similarity calculation unit, a first deduplication unit, and a second deduplication unit. The first preprocessing unit is used to perform text preprocessing on each of the first error logs to obtain a first preprocessed text corresponding to each of the first error logs. The second preprocessing unit is used to perform text preprocessing on each of the second error logs to obtain a second preprocessed text corresponding to each of the second error logs. The text similarity calculation unit is used to calculate the second text similarity between each of the first preprocessed texts, and to calculate the third text similarity between each of the second preprocessed texts. The first deduplication unit is used to perform text deduplication on each of the first preprocessed texts according to each of the second text similarities to obtain at least one of the first log texts. The second deduplication unit is used to perform text deduplication on each of the second preprocessed texts according to each of the third text similarities to obtain at least one of the second log texts.

[0124] In one embodiment, the error log analysis device 200 of the present application further includes a log quantity calculation module and a second result generation module. The log quantity calculation module is configured to calculate the difference between the number of logs in the first error log and the number of logs in the second error log if at least one of the first error logs is recorded in the latest recording period. The second result generation module is configured to generate a second log analysis result reflecting whether there has been a sudden increase in error logs in the latest recording period based on the relationship between the difference and a preset threshold value.

[0125] In one embodiment, the error log analysis device 200 of the present application also includes a third query module and a third result generation module. Wherein, the third query module is used to perform text processing on each of the first error logs if the second error log is not recorded in the historical record period to obtain at least one first log text, and use each of the first log texts as a query keyword to perform log query on the third error log recorded in the verification record period, and obtain the second query result corresponding to each first log text; wherein, the end time of the verification record period is not earlier than the start time of the historical record period, and the duration of the verification record period is greater than the duration of the historical record period. The third result generation module is used to generate the first log analysis result based on each of the second query results.

[0126] In one embodiment, the first query module 210 of the present application includes a log quantity query unit and a log text query unit. The log quantity query unit is configured to call a log quantity query interface based on the latest recording period to obtain the number of logs in the first error log returned by the log quantity query interface. The log text query unit is configured to call a log text query interface based on the latest recording period to obtain each first error log returned by the log text query interface if the number of logs in the first error log is greater than or equal to zero.

[0127] In one embodiment, the error log analysis device 200 of the present application further includes a fourth result generation module configured to generate a first log analysis result reflecting that no error logs of a new error type are recorded in the latest recording period if the number of logs in the first error log is zero.

[0128] In one embodiment, the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the error log analysis method in any embodiment.

[0129] In one embodiment, the present application also provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the error log analysis method in any embodiment.

[0130] Schematically, FIG3 is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. In one example, the computer device can be a server. Referring to FIG3 , the computer device 900 includes a processing component 902, which further includes one or more processors, and a memory resource represented by a memory 901 for storing instructions that can be executed by the processing component 902, such as an application. The application stored in the memory 901 may include one or more modules, each of which corresponds to a set of instructions. In addition, the processing component 902 is configured to execute instructions to perform the steps of the error log analysis method described in any of the above embodiments.

[0131] The computer device 900 may further include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate based on an operating system stored in the memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0132] Those skilled in the art will understand that the internal structure of the computer device shown in the present application is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0133] Finally, it should be noted that, in this document, relational terms such as first and second are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or device comprising the element. As used herein, "a," "an," "said," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. A plurality refers to at least two, such as 2, 3, 5, or 8. "And / or" includes any and all combinations of the relevant listed items.

[0134] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0135] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for analyzing error logs, characterized in that, The method includes: When the current timing moment arrives, query the first error log recorded within the latest record period; the end moment of the latest record period is the current timing moment; If at least one of the first error logs is recorded within the latest record period, query the second error log recorded within the historical record period; the end moment of the historical record period is not later than the start moment of the latest record period; If at least one of the second error logs is recorded within the historical record period, perform text processing on each of the first error logs to obtain at least one first log text, and perform text processing on each of the second error logs to obtain at least one second log text; For each of the first log texts, calculate the first text similarity between the first log text and each of the second log texts, and determine whether the first log text is similar to at least one of the second log texts based on each of the first text similarities, and obtain the similarity judgment result; Generate a first log analysis result based on each of the similarity judgment results, and the first log analysis result is used to reflect whether an error log of a new error type is recorded within the latest record period.

2. The method according to claim 1, wherein The generating the first log analysis result based on each of the similarity judgment results includes: Based on each of the similarity judgment results, determine whether there is an abnormal log text; wherein, the abnormal log text is a first log text that is not similar to each of the second log texts; If there is at least one of the abnormal log texts, respectively use each of the abnormal log texts as a query keyword to query the third error log recorded within the verification record period, and obtain the first query result corresponding to each of the abnormal log texts; wherein, the end moment of the verification record period is not earlier than the start moment of the historical record period, and the duration of the verification record period is greater than the duration of the historical record period; Generate the first log analysis result according to each of the first query results.

3. The method according to claim 1, wherein The performing text processing on each of the first error logs to obtain at least one first log text, and performing text processing on each of the second error logs to obtain at least one second log text includes: Perform text preprocessing on each of the first error logs respectively to obtain the first preprocessing text corresponding to each of the first error logs; Perform text preprocessing on each of the second error logs respectively to obtain the second preprocessing text corresponding to each of the second error logs; Calculate the second text similarity between each pair of the first preprocessing texts, and calculate the third text similarity between each pair of the second preprocessing texts; Perform text deduplication on each of the first preprocessing texts according to each of the second text similarities to obtain at least one of the first log texts; Perform text deduplication on each of the second preprocessing texts according to each of the third text similarities to obtain at least one of the second log texts.

4. The method according to claim 1, characterized in that, The method further includes: If at least one of the first error logs is recorded within the latest recording period, calculate the quantity difference between the number of the first error logs and the number of the second error logs; Generate a second log analysis result for reflecting whether there is a sudden increase in error logs within the latest recording period according to the magnitude relationship between the quantity difference and a preset quantity threshold.

5. The method according to any one of claims 1 to 4, characterized in that The method further includes: If no second error log is recorded within the historical recording period, perform text processing on each of the first error logs to obtain at least one first log text, and use each of the first log texts as a query keyword to query the third error logs recorded within the verification recording period, and obtain a second query result corresponding to each of the first log texts; wherein, the end time of the verification recording period is not earlier than the start time of the historical recording period, and the duration of the verification recording period is greater than the duration of the historical recording period; Generate the first log analysis result according to each of the second query results.

6. The method according to any one of claims 1 to 4, characterized in that The querying of the first error logs recorded within the latest recording period includes: Call a log quantity query interface according to the latest recording period to obtain the number of the first error logs returned by the log quantity query interface; If the number of the first error logs is greater than or equal to zero, call a log text query interface according to the latest recording period to obtain each of the first error logs returned by the log text query interface described.

7. The method according to claim 6, characterized in that, The method further includes: If the number of the first error logs is zero, generate a first log analysis result for reflecting that no error log of a new error type is recorded within the latest recording period.

8. An error log analysis device, characterized in that, The apparatus includes: A first query module, configured to query the first error logs recorded within the latest recording period when the current timing moment arrives; the end time of the latest recording period is the current timing moment; A second query module, configured to query the second error logs recorded within the historical recording period if at least one of the first error logs is recorded within the latest recording period; the end time of the historical recording period is not later than the start time of the latest recording period; A text processing module, configured to perform text processing on each of the first error logs to obtain at least one first log text, and perform text processing on each of the second error logs to obtain at least one second log text if at least one of the second error logs is recorded within the historical recording period; A log similarity judgment module, configured to calculate a first text similarity between each first log text and each second log text respectively, and judge whether the first log text is similar to at least one of the second log texts according to each first text similarity, and obtain a similarity judgment result; A first result generation module, configured to generate a first log analysis result based on each of the similarity judgment results, and the first log analysis result is used to reflect whether an error log of a new error type is recorded within the latest recording period.

9. A storage medium, characterized in that, The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to execute the steps of the error log analysis method according to any one of claims 1 to 7.

10. A computer device, characterized in that, Comprising: One or more processors, and a memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, execute the steps of the error log analysis method according to any one of claims 1 to 7.

Images