Log analysis method and device

Abstract

The embodiment of the invention provides a log analysis method and device, and the method comprises the steps: obtaining a to-be-analyzed log and an analysis problem inputted in a natural language, inputting the analysis problem into a first large language model, carrying out the analysis through the first large language model under the guidance of a first prompt word, and obtaining a target log type needed for answering the analysis problem, screening all the to-be-analyzed logs to obtain a log line of which the log type is a target log type, generating a first compressed log according to the screened log line, and only inputting the first compressed log and the analysis problem into a second large language model, and the second large language model outputs the log analysis result of the answer analysis question under the guidance of the second cue word, so that the log analysis range is effectively reduced, unnecessary data processing is reduced, the calculation complexity is reduced, and the log analysis cost is further reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent analysis technology, and in particular to a log analysis method and device. Background Art

[0002] In order to promptly discover business anomalies in business platforms, it is usually necessary to analyze the logs in each business platform.

[0003] Manual log analysis is time-consuming and labor-intensive. Traditional logs are automatically analyzed through expert systems and other methods, relying mainly on specific rules for retrieval and matching, and their functions are relatively limited. Analyzing logs through LLM (Large Language Model) can fully leverage the advantages of large languages ​​for fast and relatively accurate analysis. However, the large language model has a limit on the number of tokens (words) when inputting logs. A token can be understood as a Chinese character or an English character. The limit on the number of tokens is determined by the structure of the model. The more tokens it supports, the more complex the model is, and the higher the computational cost required. Since logs usually have very long contexts, directly inputting these logs into the large language model will exceed the token processing limit of the large language model, resulting in a higher analysis cost. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a log analysis method and apparatus to reduce the cost of log analysis. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a log analysis method is provided, the method comprising:

[0006] Obtaining a log to be analyzed and an analysis question; wherein the analysis question is a question input in natural language;

[0007] Inputting the analysis question into a first large language model, and obtaining a target log type output by the first large language model under the guidance of a first prompt word; wherein the target log type is a log type required to answer the analysis question; and the first prompt word is used to guide the first large language model to determine the log type required to answer the analysis question;

[0008] Generate a first compressed log according to a log line whose log type in the log to be analyzed is the target log type;

[0009] The analysis question and the first compressed log are input into the second largest language model, and a target log analysis result output by the second largest language model under the guidance of a second prompt word is obtained; wherein the target log analysis result is a log analysis result that answers the analysis question; and the second prompt word is used to guide the second largest language model to answer the analysis question.

[0010] In a possible implementation, generating a first compressed log according to a log line of the target log type in the log to be analyzed includes:

[0011] Generate a second compressed log according to a log line whose log type in the log to be analyzed is the target log type;

[0012] The second compressed log is input into a third language model, and a first log line output by the third language model under the guidance of a third prompt word is obtained to generate a first compressed log; wherein the first log line is a log line summarizing each log line in the second compressed log; and the third prompt word is used to guide the third language model to summarize each log line in the second compressed log.

[0013] In a possible implementation, generating a first compressed log according to a log line of the target log type in the log to be analyzed includes:

[0014] generating a third compressed log according to a log line of the target log type in the log to be analyzed;

[0015] Obtaining log lines whose log line lengths are greater than a preset length threshold in the third compressed log, and generating a fourth compressed log; obtaining log lines whose log line lengths are not greater than the preset length threshold in the third compressed log, and generating a fifth compressed log;

[0016] Inputting the fourth compressed log into a fourth language model, obtaining a second log line output by the fourth language model under the guidance of a fourth prompt word, and generating a sixth compressed log; wherein the second log line is a log line summarizing each log line in the fourth compressed log; and the fourth prompt word is used to guide the fourth language model to summarize each log line in the fourth compressed log;

[0017] A first compressed log is generated according to the fifth compressed log and the sixth compressed log.

[0018] In a possible implementation, generating a first compressed log according to a log line of the target log type in the log to be analyzed includes:

[0019] generating a seventh compressed log according to a log line of the target log type in the log to be analyzed;

[0020] Log lines that meet a preset log condition in the seventh compressed log are obtained to generate a first compressed log; wherein the preset log condition is used to determine a log that meets the log format requirement of the second largest language model for the input log.

[0021] In a possible implementation, obtaining the log lines that meet a preset log condition in the seventh compressed log to generate the first compressed log includes:

[0022] Obtaining log lines that meet a preset log condition in the seventh compressed log to generate an eighth compressed log;

[0023] Obtaining log lines whose log line lengths are greater than the preset length threshold in the eighth compressed log to generate a ninth compressed log; obtaining log lines whose log line lengths are not greater than the preset length threshold in the eighth compressed log to generate a tenth compressed log;

[0024] Inputting the ninth compressed log into the fifth language model, obtaining a third log line output by the fifth language model under the guidance of a fifth prompt word, and generating an eleventh compressed log; wherein the third log line is a log line summarizing each log line in the ninth compressed log; and the fifth prompt word is used to guide the fifth language model to summarize each log line in the ninth compressed log;

[0025] A first compressed log is generated according to the tenth compressed log and the eleventh compressed log.

[0026] In a second aspect of the present invention, a log analysis device is provided, comprising:

[0027] An initial acquisition module is used to obtain logs to be analyzed and analysis questions; wherein the analysis questions are questions input in natural language;

[0028] a type acquisition module, configured to input the analysis question into a first large language model, and acquire a target log type output by the first large language model under the guidance of a first prompt word; wherein the target log type is the log type required to answer the analysis question; and the first prompt word is used to guide the first large language model to determine the log type required to answer the analysis question;

[0029] a log screening module, configured to generate a first compressed log according to a log line of the target log type in the log to be analyzed;

[0030] A log analysis module is used to input the analysis question and the first compressed log into a second large language model, and obtain a target log analysis result output by the second large language model under the guidance of a second prompt word; wherein the target log analysis result is a log analysis result that answers the analysis question; and the second prompt word is used to guide the second large language model to answer the analysis question.

[0031] In a possible implementation, the log screening module includes:

[0032] A first screening submodule is configured to generate a second compressed log based on log lines of the target log type in the log to be analyzed;

[0033] The second filtering submodule is configured to input the second compressed log into a third language model, obtain a first log line output by the third language model under the guidance of a third prompt word, and generate a first compressed log; wherein the first log line is a log line summarizing each log line in the second compressed log; and the third prompt word is used to guide the third language model to summarize each log line in the second compressed log.

[0034] In a possible implementation, the log screening module includes:

[0035] A third screening submodule, configured to generate a third compressed log based on log lines of the target log type in the log to be analyzed;

[0036] a fourth filtering submodule, configured to obtain log lines in the third compressed log whose lengths are greater than a preset length threshold, and generate a fourth compressed log; and obtain log lines in the third compressed log whose lengths are not greater than the preset length threshold, and generate a fifth compressed log;

[0037] a fifth filtering submodule, configured to input the fourth compressed log into a fourth language model, obtain a second log line output by the fourth language model under the guidance of a fourth prompt word, and generate a sixth compressed log; wherein the second log line is a log line summarizing each log line in the fourth compressed log; and the fourth prompt word is used to guide the fourth language model to summarize each log line in the fourth compressed log;

[0038] The sixth screening submodule is configured to generate a first compressed log based on the fifth compressed log and the sixth compressed log.

[0039] In a possible implementation, the log screening module includes:

[0040] a seventh screening submodule, configured to generate a seventh compressed log according to log lines of the target log type in the log to be analyzed;

[0041] an eighth screening submodule, configured to obtain log lines in the seventh compressed log that meet a preset log condition and generate a first compressed log; wherein the preset log condition is used to determine logs that meet the log format requirements of the second largest language model for input logs.

[0042] In a possible implementation, the eighth screening submodule includes:

[0043] a first screening unit, configured to obtain log lines that meet a preset log condition in the seventh compressed log and generate an eighth compressed log;

[0044] a second screening unit configured to obtain, from the eighth compressed log, log lines whose lengths are greater than the preset length threshold, and generate a ninth compressed log; and obtain, from the eighth compressed log, log lines whose lengths are not greater than the preset length threshold, and generate a tenth compressed log;

[0045] a third screening unit configured to input the ninth compressed log into a fifth language model, obtain a third log line output by the fifth language model under the guidance of a fifth prompt word, and generate an eleventh compressed log; wherein the third log line is a log line summarizing each log line in the ninth compressed log; and the fifth prompt word is used to guide the fifth language model to summarize each log line in the ninth compressed log;

[0046] The fourth screening unit is configured to generate a first compressed log according to the tenth compressed log and the eleventh compressed log.

[0047] In another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, any of the above-mentioned log analysis methods is implemented.

[0048] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned log analysis methods.

[0049] A log analysis method and device provided by an embodiment of the present invention obtains a log to be analyzed and an analysis question input in natural language, inputs the analysis question into a first large language model, and the first large language model analyzes and obtains a target log type required to answer the analysis question under the guidance of a first prompt word. Log lines with the target log type are screened out from all logs to be analyzed, and a first compressed log is generated based on the screened log lines. Only the first compressed log and the analysis question are input into a second large language model, and the second large language model outputs a log analysis result that answers the analysis question under the guidance of a second prompt word. This effectively narrows the scope of log analysis, reduces unnecessary data processing, reduces computational complexity, and thereby reduces the cost of log analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0051] Figure 1 This is a schematic diagram of a first flow chart of a log analysis method according to an embodiment of the present invention;

[0052] Figure 2 This is a second flow chart of the log analysis method according to an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the third flow chart of the log analysis method according to an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the fourth flow chart of the log analysis method according to an embodiment of the present invention;

[0055] Figure 5 Schematic diagram of the fifth flow chart of the log analysis method in an embodiment of the present invention;

[0056] Figure 6 Schematic diagram of the sixth flow chart of the log analysis method according to an embodiment of the present invention;

[0057] Figure 7 7 is a flow chart of the log analysis method according to an embodiment of the present invention;

[0058] Figure 8 8 is a flow chart of an eighth method for analyzing logs according to an embodiment of the present invention;

[0059] Figure 9 This is a schematic diagram of the structure of a log analysis device according to an embodiment of the present invention;

[0060] Figure 10 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

[0062] Large language models are deeply trained based on massive amounts of data. However, they have strict token limits on input. This is determined by their structure: the more tokens they support, the more complex the model becomes. Currently, large language models support a maximum of 8,000, 16,000, or 32,000 tokens per input, generally up to 128,000. If the input exceeds this limit, the model will fail to process it. Furthermore, even within this limit, the greater the number of tokens inputted, the higher the computational costs, including computing power and the cost of invoking third-party models.

[0063] However, logs typically have a long context. For example, a user playback event in a video app involves parameters passed to the external call to start the playback, the playback process, user operations during playback, and interactions between various modules within the player. This short playback event can generate a large amount of logs, often thousands, tens of thousands, or even hundreds of thousands of lines. If these logs were fed into a large model for log analysis, the number of tokens the model can handle would far exceed its capacity.

[0064] In existing log analysis, some methods can also be used to pre-process or post-process the logs to minimize the amount of log content, such as truncating logs that exceed a certain length and filtering low-priority logs. Although these methods can reduce some of the log content, they may cause the logs that are ultimately input into the large model to lack some of the truly required log content, affecting the input of the large model and resulting in low accuracy of the log analysis results.

[0065] Based on this, the embodiment of the present invention provides a log analysis method, see Figure 1 , Figure 1 This is a schematic diagram of a first flow chart of a log analysis method according to an embodiment of the present invention, the method comprising:

[0066] S101, obtaining the log to be analyzed and the analysis problem.

[0067] The analysis question is a question input in natural language.

[0068] S102: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0069] The target log type is the log type required to answer the analysis question. The first prompt word is used to guide the first language model to determine the log type required to answer the analysis question.

[0070] S103: Generate a first compressed log according to log lines whose log type is the target log type in the log to be analyzed.

[0071] S104: Input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0072] The target log analysis result is a log analysis result that answers the analysis question. The second prompt word is used to guide the second language model to answer the analysis question.

[0073] By applying the above embodiment, by obtaining the log to be analyzed and the analysis question input in natural language, the analysis question is input into the first largest language model, and the first largest language model analyzes and obtains the target log type required to answer the analysis question under the guidance of the first prompt word. Log lines with the target log type are screened out from all the logs to be analyzed, and a first compressed log is generated based on the screened log lines. Only the first compressed log and the analysis question are input into the second largest language model, and the second largest language model outputs the log analysis result that answers the analysis question under the guidance of the second prompt word, which effectively narrows the scope of log analysis, reduces unnecessary data processing, reduces computational complexity, and thereby reduces the cost of log analysis.

[0074] In addition, since log lines irrelevant to the analysis problem (ie irrelevant to the current log analysis) are filtered out during the log screening process, only log content relevant to the analysis problem is retained, thereby improving the accuracy of log analysis.

[0075] Steps S101 to S104 are described in detail below:

[0076] In step S101, the logs to be analyzed are detailed data such as various events, state changes, and error messages during the operation of computer systems, network devices, applications, etc. Exemplarily, the logs to be analyzed include but are not limited to video playback logs, software application logs, system logs, network logs, hardware logs, etc.

[0077] The video playback log records various events during video playback, such as playback start, pause, resume, and end, as well as errors or abnormal situations that may be encountered during playback, such as video loading failure, freezes, etc. The software application log records the running status of the software application in detail, including events such as startup, shutdown, user operations, system calls, and error messages, warning messages, or debugging information generated by the application during operation. The system log records various events at the operating system level, such as system startup, shutdown, user login, logout, etc., as well as system resource usage, security events, etc. The network log records the running status of network devices, changes in network traffic, network connection establishment and disconnection, and other events, as well as possible network attacks or abnormal behaviors. The hardware log records the running status, temperature, voltage and other parameters of computer hardware devices, as well as possible failure or alarm information of hardware devices, which are not listed here in detail.

[0078] An analysis question is a question that a user enters in natural language, specifically about the log being analyzed, when they want to obtain a specific analysis result. Analysis questions can be predefined or user-entered in real time, and are not specifically limited here. For example, analysis questions could be, "Why didn't the video start successfully this time?", "Why didn't the software start successfully this time?", or "Analyze why the network connection was disconnected?" The examples are not exhaustive here.

[0079] In step S102 , the first large language model is a large language model that can output the log type required to answer the analysis question based on the input analysis question under the guidance of the first prompt word.

[0080] Each log line in the log to be analyzed has an explicit tag (TAG) or identifier that indicates the log type of the log line. The log type can be the default type when the system generates the log, such as INFO (information), DEBUG (debug), ERROR (error), and WARN (warning). It can also be a type defined by the log business party when outputting the log. For example, for a video player application, when outputting the log, the corresponding type will be added to each log, such as Video (video), Codec (coder / decoder, encoder and decoder), Audio (audio), and Surface (hardware device brand). Different TAG types will be filtered according to the specific problem during log analysis.

[0081] For example, taking the video playback of video software, the log type is the default type when the system generates logs as an example. The user finds that the video cannot be played normally, and needs to locate the reason why the video cannot be played normally by analyzing the logs during the video playback process.

[0082] For example, the log to be analyzed is as follows:

[0083] "1.[INFO][2023-10-05 14:30:00]System startup initialized.

[0084] 2.[DEBUG][2023-10-05 14:30:10]Loading configuration file:config.json.

[0085] 3.[ERROR][2023-10-05 14:30:11]Video streaming service failed to initialize:missing required credential.

[0086] 4.[INFO][2023-10-05 14:30:40]Audio encoder initialized successfully.

[0087] 5.[DEBUG][2023-10-05 14:31:00]Connecting to video server at ip:192.168.1.10,port:8080.

[0088] 6.[WARN][2023-10-05 14:32:00]Network latency detected:150ms.

[0089] 7.[ERROR][2023-10-05 14:32:50]Failed to start video stream: video streaming service not ready.”

[0090] For the convenience of description, they are recorded from top to bottom as log line 1, log line 2, log line 3, log line 4, log line 5, log line 6 and log line 7.

[0091] Log lines 1 and 4 are of type INFO, or informational logs, which record key events during the normal operation of the program. Log line 1 indicates system startup initialization, while log line 4 indicates successful initialization of the audio encoder.

[0092] Log lines 2 and 5 are of the DEBUG type, meaning debug logs, which record debugging information during program execution. Log line 2 indicates debugging information during program execution, such as configuration file loading, while log line 5 records detailed information about the connection to the video server.

[0093] Log lines 3 and 7 are of the ERROR type, meaning error logs, which record errors that affect program execution. Log line 3 indicates that the video streaming service failed to initialize due to a lack of necessary credentials, while log line 7 indicates that the video streaming service failed to start due to the service not being ready.

[0094] The log type of log line 6 is WARN, which is a warning log. It is used to record error conditions that occur during program execution but do not affect program operation. Log line 6 is used to indicate that a network delay of 150ms was detected.

[0095] Based on the log to be analyzed, the user wants to analyze the log to determine why the video failed to play. The input analysis question is "Why didn't this video play successfully?" This analysis question is input to the first language model, and the output answers the analysis question "Why didn't this video play successfully?" The required log types are ERROR and WARN.

[0096] In step S103 , it can be understood that a log is composed of multiple log lines. Therefore, generating a compressed log based on log lines in this article refers to using a log line set composed of log lines as a compressed log.

[0097] Still taking the above example, according to the log lines of the target log type in the log to be analyzed, the generated first compressed log includes log line 3, log line 6, and log line 7.

[0098] In step S104, the second largest language model is a large language model enhanced by a knowledge base that contains the business knowledge required for analyzing compressed logs. Based on this, the second largest language model can output log analysis results that answer the analysis question based on the input analysis question and the compressed log, guided by the second prompt word.

[0099] For example, still taking the above example, the analysis question "Why did the video fail to start broadcasting successfully this time?" and the first compressed log "log line 3, log line 6 and log line 7" are input into the second largest language model to obtain the log analysis result "Video streaming service is not ready" output by the model.

[0100] It can be understood that the above-mentioned logs to be analyzed, analysis questions, log types and log analysis results are only one possible embodiment provided by the present invention. In other embodiments, the logs to be analyzed, analysis questions, log types and log analysis results can also be other examples, which are not specifically limited here.

[0101] In order to further optimize the log analysis process and improve the efficiency of log analysis, in a possible embodiment, the first compressed log can also be compressed twice, see Figure 2 , Figure 2 This is a second flow chart of the log analysis method according to an embodiment of the present invention, which includes:

[0102] S201, obtaining the log to be analyzed and the analysis problem.

[0103] The analysis question is a question input in natural language.

[0104] S202: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0105] The target log type is the log type required to answer the analysis question. The first prompt word is used to guide the first language model to determine the log type required to answer the analysis question.

[0106] S203: Generate a second compressed log according to the log lines whose log type is the target log type in the log to be analyzed.

[0107] S204: Input the second compressed log into the third language model, obtain the first log line output by the third language model under the guidance of the third prompt word, and generate a first compressed log.

[0108] The first log line is a log line that summarizes each log line in the second compressed log, and the third prompt word is used to guide the third language model to summarize each log line in each second compressed log.

[0109] S205 , inputting the analysis question and the first compressed log into the second largest language model, and obtaining a target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0110] Among them, the target log analysis result is the log analysis result that answers the analysis question, and the second prompt word is used to guide the second largest language model to answer the analysis question.

[0111] Steps S201 to S205 are described in detail below:

[0112] Steps S201-S202 are equivalent to the aforementioned steps S101-S102, and step S205 is equivalent to the aforementioned step S104. The detailed descriptions of steps S201-S202 and step S205 can be found in the descriptions of steps S101-S102 and step S104, and will not be repeated here. Steps S202-S203 are the process of filtering logs based on their type, so steps S202-S203 can be referred to as type filtering. Step S204 is the process of summarizing each log line in the log, so step S204 can be referred to as summarization processing.

[0113] In step S203, a second compressed log is generated based on the log lines in the log to be analyzed whose log type is the target log type. That is, in this embodiment, the second compressed log is a log that has been type-filtered.

[0114] In step S204, the second compressed log is input into the third language model, and the first log line output by the third language model under the guidance of the third prompt word is obtained, thereby generating a first compressed log. That is, the first compressed log is a log that has been type-filtered and summarized. The third language model is also a large language model enhanced by a knowledge base, which contains the business knowledge required to analyze the compressed log. Based on this, the third language model can understand the meaning of each log line in the second compressed log and output a first log line obtained by summarizing each log line in the second compressed log. Based on each first log line, a first compressed log is generated.

[0115] For example, the original log behavior is as follows: [2024-07-15 14:30:22][ERROR][UserService] When the user with user ID 12345 logged into the system, the account lock mechanism was triggered because the number of incorrect passwords exceeded 3 times, and the current operation was rejected by the system.

[0116] The third language model analyzes the original log line to understand its meaning: it records the time, error type, service module, and information about the user's account being locked and the operation being denied due to exceeding the password error limit. The time, error type, user ID, and the reason that triggered the account lockout are key information.

[0117] The third language model summarizes the original log line and obtains the first log behavior: [2024-07-15 14:30:22][ERROR][UserService] User 12345's account was locked due to an incorrect password exceeding the limit, and the operation was denied.

[0118] In step S205, the analysis text and the first compressed log obtained by secondary compression by the third language model are input into the second language model to obtain the log analysis result of the answer analysis question output by the second language model.

[0119] By applying the above embodiment, the compressed log that has been type-filtered is summarized by introducing the third language model, that is, secondary compression is performed to generate a more refined first compressed log. When the analysis problem is combined with the log data for problem analysis, the first compressed log (rather than the compressed log that has only been type-filtered) is used as input. This step effectively reduces the redundant information of the log data and reduces the number of tokens input to the model, thereby reducing the amount of data required to be processed by the second language model, that is, reducing the analysis cost. Since the process of summarizing the log is to extract the key information in each log line, it can output the log analysis results more quickly while maintaining the integrity of the key information, which not only optimizes the log analysis process and improves the efficiency of log analysis, but also improves the accuracy of log analysis.

[0120] Based on the above embodiment, in order to reduce the number of tokens input to the large language model for summarizing logs and further improve the efficiency of log analysis, in a possible embodiment, the logs input to the large language model for summarizing logs can be filtered, and only the logs with log line lengths greater than a preset length threshold are input to the large language model for summarizing logs. Figure 3 , Figure 3 This is a third flow chart of the log analysis method according to an embodiment of the present invention, which includes:

[0121] S301, obtaining the log to be analyzed and the analysis problem.

[0122] The analysis question is a question input in natural language.

[0123] S302: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0124] The target log type is the log type required to answer the analysis question. The first prompt word is used to guide the first language model to determine the log type required to answer the analysis question.

[0125] S303: Generate a third compressed log according to the log lines whose log type is the target log type in the log to be analyzed.

[0126] S304 , obtaining log lines in the third compressed log whose length is greater than a preset length threshold, and generating a fourth compressed log; obtaining log lines in the third compressed log whose length is not greater than the preset length threshold, and generating a fifth compressed log.

[0127] S305 : Input the fourth compressed log into the fourth language model, obtain the second log line output by the fourth language model under the guidance of the fourth prompt word, and generate a sixth compressed log.

[0128] The second log line is a log line that summarizes each log line in the fourth compressed log. The fourth prompt word is used to guide the fourth language model to summarize each log line in the fourth compressed log.

[0129] S306: Generate a first compressed log according to the fifth compressed log and the sixth compressed log.

[0130] S307: Input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0131] The target log analysis result is the log analysis result that answers the analysis question. The second prompt word is used to guide the second language model to answer the analysis question.

[0132] The following will describe steps S301 to S307 in detail:

[0133] Steps S301-S302 are equivalent to the aforementioned steps S101-S102, and step S307 is equivalent to the aforementioned step S104. The specific descriptions of steps S301-S302 and step S307 can be found in the descriptions of the aforementioned steps S101-S102 and step S104, and will not be repeated here.

[0134] In step S303, a third compressed log is generated based on the log lines of the target log type in the log to be analyzed. That is, in this embodiment, the third compressed log is the same as the second compressed log, and both are logs that have been type-filtered.

[0135] In step S304, the preset length threshold should be a clear dividing line between long log lines and short log lines, which can be set by professional technicians based on work experience or industry standards, and is not specifically limited here.

[0136] In step S305, since log lines with lengths no greater than the preset length threshold are relatively short, directly inputting them into the second largest language model will not significantly affect the number of tokens. However, log lines with lengths greater than the preset length threshold are longer, significantly impacting the number of tokens. Therefore, their lengths need to be shortened. Specifically, only log lines with lengths greater than the preset length threshold are obtained from the third compressed log to generate a fourth compressed log. The fourth compressed log is then input into the fourth largest language model. Guided by a fourth prompt, the fourth largest language model summarizes the content of each log line in the second compressed log and outputs the second log line, thereby shortening its length. The fourth largest language model functions similarly to the third largest language model: both are large language models used to summarize log lines and thereby shorten their length. Log lines with lengths no greater than the preset length threshold are obtained to generate a fifth compressed log, eliminating the need to further input them into the fourth largest language model for summarization to shorten their lengths.

[0137] In step S306 , a first compressed log is generated based on the sixth compressed log obtained by summarizing the fourth language model and the fifth compressed log that does not require shortening of log line lengths.

[0138] For ease of understanding, the present invention provides the foregoing embodiments. Figure 3 Another form of flow chart, see Figure 4 , Figure 4 This is a fourth flow chart of a log analysis method according to an embodiment of the present invention, which includes:

[0139] S401, obtaining the log to be analyzed and the analysis problem.

[0140] The analysis question is a question input in natural language.

[0141] S402: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0142] The target log type is the log type required to answer the analysis question. The first prompt word is used to guide the first language model to determine the log type required to answer the analysis question.

[0143] S403: Generate a third compressed log according to the log lines whose log type is the target log type in the log to be analyzed.

[0144] S404: Determine whether the length of the log line in the third compressed log is greater than a preset length threshold. If so, execute step S405; if not, execute step S407.

[0145] S405: Obtain log lines in the third compressed log whose lengths are greater than a preset length threshold, and generate a fourth compressed log.

[0146] S406 : Input the fourth compressed log into the fourth language model, obtain the second log line output by the fourth language model under the guidance of the fourth prompt word, and generate a sixth compressed log.

[0147] S407: Obtain log lines in the third compressed log whose lengths are not greater than a preset length threshold, and generate a fifth compressed log.

[0148] S408: Generate a first compressed log according to the fifth compressed log and the sixth compressed log.

[0149] S409: Input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0150] The target log analysis result is the log analysis result that answers the analysis question. The second prompt word is used to guide the second language model to answer the analysis question.

[0151] Steps S401 to S409 are equivalent to the aforementioned steps S301 to S307. The specific descriptions of steps S401 to S409 can be found in the descriptions of the aforementioned steps S301 to S307, which will not be repeated here.

[0152] Applying the above embodiment, the log is initially compressed using the selected target log type log lines to obtain a third compressed log. The third compressed log is then further processed based on the log line length. Log lines whose length is no greater than a preset length threshold are obtained to generate a fifth compressed log. Log lines whose length is greater than the preset length threshold are obtained to generate a fourth compressed log. These are then input into a fourth language model and summarized using the fourth language model to generate a sixth compressed log. A first compressed log is generated based on the fifth and sixth compressed logs, and both are input into the second language model to obtain log analysis results that answer the analysis question. This not only reduces the complexity of data processing but also improves the integrity of the actual log content, thereby optimizing the log analysis process and improving the efficiency and accuracy of log analysis.

[0153] As in the above specific example, the log to be analyzed includes not only the log content, but also the format content, such as time information, symbol information, etc. Since the format content is not the substantive log content, it has little impact on the analysis results. Therefore, in order to make the compressed log input to the second language model contain only the substantive log content and reduce the data processing amount of the model, the format content in the log can be deleted before inputting it to the second language model. See Figure 5 , Figure 5 This is a fifth flow chart of a log analysis method according to an embodiment of the present invention, which includes:

[0154] S501: Obtain the log to be analyzed and the analysis problem.

[0155] The analysis question is a question input in natural language.

[0156] S502: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0157] S503: Generate a seventh compressed log according to the log lines whose log type is the target log type in the log to be analyzed.

[0158] S504: Obtain log lines that meet a preset log condition in the seventh compressed log, and generate a first compressed log.

[0159] The preset log condition is used to determine logs that meet the log format requirements of the second largest language model for input logs.

[0160] S505 : Input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0161] The target log analysis result is the log analysis result that answers the analysis question. The second prompt word is used to guide the second language model to answer the analysis question.

[0162] Steps S501-S502 are equivalent to the aforementioned steps S101-S102, and step S505 is equivalent to the aforementioned step S104. The specific descriptions of steps S501-S502 and step S505 can be found in the descriptions of the aforementioned steps S101-S102 and step S104, which will not be repeated here.

[0163] In step S203, a seventh compressed log is generated based on the log lines of the target log type in the log to be analyzed. That is, in this embodiment, the seventh compressed log is the same as the second and third compressed logs, both of which are type-filtered logs.

[0164] In step S504, log lines that meet a preset log condition in the seventh compressed log are obtained to generate a first compressed log, wherein the preset log condition is used to determine a log that meets the log format requirement of the second largest language model for the input log.

[0165] Log format requirements may vary in different log analysis scenarios. For example, the log format requirement may include no time information, no symbol information, or both. In other embodiments, the log format requirement may be other examples, which are not specifically limited here. Obtain log lines in the seventh compressed log that meet the preset log conditions, that is, by deleting the content in the log lines of the seventh compressed log that does not meet the log format requirements, obtaining the log lines in the seventh compressed log that meet the preset log conditions.

[0166] Taking the aforementioned log line 6 as an example, the log content in log line 6 is "Network latency detected: 150ms.". The content included therein does not conform to the log format requirements, including the time information "2023-10-05 14:32:00" and the symbol information "[]". After deleting the content that does not conform to the log format requirements, the resulting log line is "WARN Network latency detected: 150ms."

[0167] In step S505, the first compressed log that meets the log format requirements is input into the second largest language model, and the second largest language model, under the guidance of the second prompt word, outputs a log analysis result that answers the analysis question based on the input analysis question and the compressed log.

[0168] By applying the above embodiment, a first compressed log is generated by obtaining log lines in the seventh compressed log that meet the preset log conditions, and a first compressed log that is more concise and only contains substantial log content is obtained. The first compressed log and the analysis question are input into the second largest language model to obtain a log analysis result that answers the analysis question, further reducing the complexity of data processing, improving the integrity of the substantial log content, and thereby optimizing the log analysis process and improving the efficiency and accuracy of log analysis.

[0169] Based on the above embodiment, the complexity of data processing can be reduced and the efficiency and accuracy of log analysis can be improved by deleting the content in the log that does not meet the log format requirements. Alternatively, the key content in the log line can be extracted by summarizing the log lines whose length is greater than the preset length threshold, thereby reducing the complexity of data processing and improving the efficiency and accuracy of log analysis. In another possible embodiment, the above two methods can be used at the same time to further reduce the complexity of data processing and improve the efficiency and accuracy of log analysis. Figure 6 , Figure 6 6 is a flow chart of a log analysis method according to an embodiment of the present invention, which includes:

[0170] S601: Obtain the log to be analyzed and the analysis problem.

[0171] The analysis question is a question input in natural language.

[0172] S602: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0173] The target log type is the log type required to answer the analysis question. The first prompt word is used to guide the first language model to determine the log type required to answer the analysis question.

[0174] S603: Generate a seventh compressed log according to the log lines whose log type is the target log type in the log to be analyzed.

[0175] S604: Obtain log lines that meet a preset log condition in the seventh compressed log, and generate an eighth compressed log.

[0176] S605 , obtaining log lines whose lengths are greater than a preset length threshold in the eighth compressed log, and generating a ninth compressed log; obtaining log lines whose lengths are not greater than the preset length threshold in the eighth compressed log, and generating a tenth compressed log.

[0177] S606: Input the ninth compressed log into the fifth language model, obtain the third log line output by the fifth language model under the guidance of the fifth prompt word, and generate an eleventh compressed log.

[0178] The third log line is a log line that summarizes each log line in the ninth compressed log. The fifth prompt word is used to guide the fifth language model to summarize each log line in the ninth compressed log.

[0179] S607: Generate a first compressed log according to the tenth compressed log and the eleventh compressed log.

[0180] S608: Input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0181] The target log analysis result is the log analysis result that answers the analysis question. The second prompt word is used to guide the second language model to answer the analysis question.

[0182] Steps S601 and S602 are equivalent to the aforementioned steps S101 and S102, step S603 is equivalent to the aforementioned step S503, step S604 is equivalent to the aforementioned step S504, and steps S605 to S608 are equivalent to the aforementioned steps S304 to S307. For a detailed description of steps S601 to S602, refer to the description of steps S101 to S102, for a detailed description of step S603, refer to the description of step S503, and for a detailed description of steps S605 to S608, refer to the description of steps S304 to S307, and are not repeated here.

[0183] For ease of understanding, the present invention provides the foregoing embodiments. Figure 6 Another form of flow chart, see Figure 7 , Figure 7 : This is a seventh flow chart of a log analysis method according to an embodiment of the present invention. The method includes:

[0184] S701: Obtain the log to be analyzed and analyze the problem.

[0185] The analysis question is a question input in natural language.

[0186] S702: Input the analysis question into the first language model, and obtain the target log type output by the first language model under the guidance of the first prompt word.

[0187] The target log type is the log type required to answer the analysis question. The first prompt word is used to guide the first language model to determine the log type required to answer the analysis question.

[0188] S703: Generate a seventh compressed log according to the log lines whose log type is the target log type in the log to be analyzed.

[0189] S704 , obtaining log lines that meet a preset log condition in the seventh compressed log, and generating an eighth compressed log.

[0190] S705 , determining whether the length of the log line in the eighth compressed log is greater than a preset length threshold; if so, executing step S706 ; if not, executing step S708 .

[0191] S706 , obtaining log lines in the eighth compressed log whose lengths are greater than a preset length threshold, and generating a ninth compressed log.

[0192] S707 : Input the ninth compressed log into the fifth language model, obtain the third log line output by the fifth language model under the guidance of the fifth prompt word, and generate an eleventh compressed log.

[0193] S708 : Obtain log lines in the eighth compressed log whose lengths are not greater than a preset length threshold, and generate a tenth compressed log.

[0194] S709: Generate a first compressed log according to the tenth compressed log and the eleventh compressed log.

[0195] S710: Input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of the second prompt word.

[0196] The target log analysis result is the log analysis result that answers the analysis question. The second prompt word is used to guide the second language model to answer the analysis question.

[0197] Steps S701 to S710 are equivalent to the aforementioned steps S601 to S608. The specific description of steps S701 to S710 can be found in the description of the aforementioned steps S601 to S608, which will not be repeated here.

[0198] The process of processing the log to be analyzed in the aforementioned step S702 can be recorded as module compression (module ecompress). Based on this, the first large language model can be recorded as a module large language model (module_LL M).

[0199] In the aforementioned step S703 , the log obtained by screening according to the log type (category_list) output by the first language model can be recorded as the log after module compression (log_after_module_compress).

[0200] The process of processing the log in the aforementioned steps S704-S709 can be recorded as line compression (linecompress). Based on this, the log line (log_line) is processed in a traditional way to delete the format content that does not meet the log format requirements. The resulting log can be recorded as a log after format content processing (log_line_after_format). The log line length greater than the preset length threshold (line_max_length) is input into the fifth language model for further compression processing. Therefore, the fifth language model has the same function as the aforementioned fourth language model and the third language model, and is a large language model used to summarize the log line and thus shorten the length of the log line. Therefore, the fifth language model, the fourth language model and the third language model can be recorded as line large language models (log_line_LLM). The resulting log can be recorded as a log after line compression (log_after_line_compress).

[0201] In the aforementioned step S710, the analyzed question and the compressed first compressed log are input into the second largest language model to obtain a log analysis result. Therefore, the second largest language model can be recorded as a question processing large language model (question_LLM).

[0202] Based on this, the process of the above log analysis method can be described as the following steps, see Figure 8 , Figure 8 The eighth flow chart of the log analysis method according to the embodiment of the present invention includes:

[0203] Step 1: Enter the problem and log.

[0204] Step 2: Module compression.

[0205] Step 3: Row compression.

[0206] Step 4: Analyze and solve the problem.

[0207] Step 5: Output analysis results.

[0208] The aforementioned step 1 is equivalent to the aforementioned step S101, or step S201, or step S301, or step S401, or step S501, or step S601, or step S701. The aforementioned step 2 is equivalent to the aforementioned steps S102-step S103, or steps S202-step S203, or steps S302-step S303, or steps S402-step S403, or steps S502-step S503, or steps S602-step S603, or steps S702-step S703. The aforementioned step 3 is equivalent to the aforementioned step S204, or steps S304-step S306, or steps S404-step S408. The aforementioned step 4 is equivalent to "inputting the analysis question and the first compressed log into the second largest language model" in the aforementioned step S104, or step S205, or step S307, or step S409, or step S505, or step S608, or step S710. The aforementioned step 5 is equivalent to "obtaining the target log analysis result output by the second largest language model under the guidance of the second prompt word" in the aforementioned step S104, or step S205, or step S307, or step S409, or step S505, or step S608, or step S710. Inputting the log in the aforementioned step 1 refers to the need to use the log to be analyzed during the log analysis process, rather than inputting the log to be analyzed into the first largest language model.

[0209] By applying the above embodiment, by accurately matching the target log type required for the analysis text, the logs related to the analysis text in the logs to be analyzed can be screened out, the interference of irrelevant log data is avoided, the content in the log lines that do not meet the log format requirements is deleted, the log content is effectively simplified, and the essential log content in the log is retained. The log lines with deleted content that does not meet the log format requirements are classified according to the length threshold. For shorter log lines, no secondary compression is performed, while for longer log lines, they are summarized through the fifth language model, that is, secondary compression is performed. Finally, the analysis problem, the short log containing only the essential content, and the long log containing only the essential content and summarized are input into the second language model to obtain the target log analysis result. This not only reduces the complexity of data processing, but also improves the integrity of the essential log content, thereby optimizing the log analysis process and improving the efficiency and accuracy of log analysis.

[0210] Corresponding to the aforementioned log analysis method, an embodiment of the present invention further provides a log analysis device, see Figure 9 , Figure 9 This is a schematic diagram of the structure of a log analysis device in an embodiment of the present invention, which includes:

[0211] Initial acquisition module 901 is used to obtain the log to be analyzed and the analysis question; wherein the analysis question is a question input in natural language;

[0212] Type acquisition module 902 is configured to input the analysis question into a first large language model and acquire a target log type output by the first large language model under the guidance of a first prompt word; wherein the target log type is the log type required to answer the analysis question; and the first prompt word is used to guide the first large language model to determine the log type required to answer the analysis question.

[0213] A log screening module 903 is configured to generate a first compressed log based on log lines in the log to be analyzed whose log type is the target log type;

[0214] The log analysis module 904 is used to input the analysis question and the first compressed log into the second largest language model, and obtain the target log analysis result output by the second largest language model under the guidance of a second prompt word; wherein the target log analysis result is the log analysis result that answers the analysis question; and the second prompt word is used to guide the second largest language model to answer the analysis question.

[0215] By applying the above embodiment, by obtaining the log to be analyzed and the analysis question input in natural language, the analysis question is input into the first largest language model, and the first largest language model analyzes and obtains the target log type required to answer the analysis question under the guidance of the first prompt word. Log lines with the target log type are screened out from all the logs to be analyzed, and a first compressed log is generated based on the screened log lines. Only the first compressed log and the analysis question are input into the second largest language model, and the second largest language model outputs the log analysis result that answers the analysis question under the guidance of the second prompt word, which effectively narrows the scope of log analysis, reduces unnecessary data processing, reduces computational complexity, and thereby reduces the cost of log analysis.

[0216] In a possible implementation, the log screening module includes:

[0217] A first screening submodule is configured to generate a second compressed log based on log lines of the target log type in the log to be analyzed;

[0218] The second filtering submodule is configured to input the second compressed log into a third language model, obtain a first log line output by the third language model under the guidance of a third prompt word, and generate a first compressed log; wherein the first log line is a log line summarizing each log line in the second compressed log; and the third prompt word is used to guide the third language model to summarize each log line in the second compressed log.

[0219] In a possible implementation, the log screening module includes:

[0220] A third screening submodule, configured to generate a third compressed log based on log lines of the target log type in the log to be analyzed;

[0221] a fourth filtering submodule, configured to obtain log lines in the third compressed log whose lengths are greater than a preset length threshold, and generate a fourth compressed log; and obtain log lines in the third compressed log whose lengths are not greater than the preset length threshold, and generate a fifth compressed log;

[0222] a fifth filtering submodule, configured to input the fourth compressed log into a fourth language model, obtain a second log line output by the fourth language model under the guidance of a fourth prompt word, and generate a sixth compressed log; wherein the second log line is a log line summarizing each log line in the fourth compressed log; and the fourth prompt word is used to guide the fourth language model to summarize each log line in the fourth compressed log;

[0223] The sixth screening submodule is configured to generate a first compressed log based on the fifth compressed log and the sixth compressed log.

[0224] In a possible implementation, the log screening module includes:

[0225] a seventh screening submodule, configured to generate a seventh compressed log according to log lines of the target log type in the log to be analyzed;

[0226] an eighth screening submodule, configured to obtain log lines in the seventh compressed log that meet a preset log condition and generate a first compressed log; wherein the preset log condition is used to determine logs that meet the log format requirements of the second largest language model for input logs.

[0227] In a possible implementation, the eighth screening submodule includes:

[0228] a first screening unit, configured to obtain log lines that meet a preset log condition in the seventh compressed log and generate an eighth compressed log;

[0229] a second screening unit configured to obtain, from the eighth compressed log, log lines whose lengths are greater than the preset length threshold, and generate a ninth compressed log; and obtain, from the eighth compressed log, log lines whose lengths are not greater than the preset length threshold, and generate a tenth compressed log;

[0230] a third screening unit configured to input the ninth compressed log into a fifth language model, obtain a third log line output by the fifth language model under the guidance of a fifth prompt word, and generate an eleventh compressed log; wherein the third log line is a log line summarizing each log line in the ninth compressed log; and the fifth prompt word is used to guide the fifth language model to summarize each log line in the ninth compressed log;

[0231] The fourth screening unit is configured to generate a first compressed log according to the tenth compressed log and the eleventh compressed log.

[0232] The embodiment of the present invention further provides an electronic device, such as Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0233] Memory 1003, used for storing computer programs;

[0234] The processor 1001 is configured to execute the program stored in the memory 1003 by performing the following steps:

[0235] Obtaining a log to be analyzed and an analysis question; wherein the analysis question is a question input in natural language;

[0236] Inputting the analysis question into a first large language model, and obtaining a target log type output by the first large language model under the guidance of a first prompt word; wherein the target log type is a log type required to answer the analysis question; and the first prompt word is used to guide the first large language model to determine the log type required to answer the analysis question;

[0237] Generate a first compressed log according to a log line whose log type in the log to be analyzed is the target log type;

[0238] The analysis question and the first compressed log are input into the second largest language model, and a target log analysis result output by the second largest language model under the guidance of a second prompt word is obtained; wherein the target log analysis result is a log analysis result that answers the analysis question; and the second prompt word is used to guide the second largest language model to answer the analysis question.

[0239] The communication bus mentioned in the terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0240] The communication interface is used for communication between the above terminal and other devices.

[0241] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0242] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0243] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the log analysis method described in any one of the above embodiments is implemented.

[0244] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the log analysis method described in any one of the above embodiments.

[0245] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0246] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0247] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0248] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A log analysis method, characterized in that: The method comprises: Obtaining a log to be analyzed and an analysis question; wherein the analysis question is a question input in natural language; Inputting the analysis question into a first large language model, and obtaining a target log type output by the first large language model under the guidance of a first prompt word; wherein the target log type is a log type required to answer the analysis question; and the first prompt word is used to guide the first large language model to determine the log type required to answer the analysis question; Generate a first compressed log according to a log line whose log type in the log to be analyzed is the target log type; The analysis question and the first compressed log are input into the second largest language model, and a target log analysis result output by the second largest language model under the guidance of a second prompt word is obtained; wherein the target log analysis result is a log analysis result that answers the analysis question; and the second prompt word is used to guide the second largest language model to answer the analysis question.

2. The method according to claim 1, characterized in that The step of generating a first compressed log according to a log line of the target log type in the log to be analyzed includes: Generate a second compressed log according to a log line whose log type in the log to be analyzed is the target log type; The second compressed log is input into a third language model, and a first log line output by the third language model under the guidance of a third prompt word is obtained to generate a first compressed log; wherein the first log line is a log line summarizing each log line in the second compressed log; and the third prompt word is used to guide the third language model to summarize each log line in the second compressed log.

3. The method according to claim 1, characterized in that The step of generating a first compressed log according to a log line of the target log type in the log to be analyzed includes: generating a third compressed log according to a log line of the target log type in the log to be analyzed; Obtaining log lines whose log line lengths are greater than a preset length threshold in the third compressed log, and generating a fourth compressed log; obtaining log lines whose log line lengths are not greater than the preset length threshold in the third compressed log, and generating a fifth compressed log; Inputting the fourth compressed log into a fourth language model, obtaining a second log line output by the fourth language model under the guidance of a fourth prompt word, and generating a sixth compressed log; wherein the second log line is a log line summarizing each log line in the fourth compressed log; and the fourth prompt word is used to guide the fourth language model to summarize each log line in the fourth compressed log; A first compressed log is generated according to the fifth compressed log and the sixth compressed log.

4. The method according to claim 1, wherein The step of generating a first compressed log according to a log line of the target log type in the log to be analyzed includes: generating a seventh compressed log according to a log line of the target log type in the log to be analyzed; Log lines that meet a preset log condition in the seventh compressed log are obtained to generate a first compressed log; wherein the preset log condition is used to determine a log that meets the log format requirement of the second largest language model for the input log.

5. The method according to claim 4, characterized in that The obtaining of the log lines that meet the preset log condition in the seventh compressed log to generate the first compressed log includes: Obtaining log lines that meet a preset log condition in the seventh compressed log to generate an eighth compressed log; Obtaining log lines whose log line lengths are greater than the preset length threshold in the eighth compressed log to generate a ninth compressed log; obtaining log lines whose log line lengths are not greater than the preset length threshold in the eighth compressed log to generate a tenth compressed log; Inputting the ninth compressed log into the fifth language model, obtaining a third log line output by the fifth language model under the guidance of a fifth prompt word, and generating an eleventh compressed log; wherein the third log line is a log line summarizing each log line in the ninth compressed log; and the fifth prompt word is used to guide the fifth language model to summarize each log line in the ninth compressed log; A first compressed log is generated according to the tenth compressed log and the eleventh compressed log.

6. A log analysis device, characterized in that: The device comprises: An initial acquisition module is used to obtain logs to be analyzed and analysis questions; wherein the analysis questions are questions input in natural language; a type acquisition module, configured to input the analysis question into a first large language model, and acquire a target log type output by the first large language model under the guidance of a first prompt word; wherein the target log type is the log type required to answer the analysis question; and the first prompt word is used to guide the first large language model to determine the log type required to answer the analysis question; a log screening module, configured to generate a first compressed log according to a log line of the target log type in the log to be analyzed; A log analysis module is used to input the analysis question and the first compressed log into a second large language model, and obtain a target log analysis result output by the second large language model under the guidance of a second prompt word; wherein the target log analysis result is a log analysis result that answers the analysis question; and the second prompt word is used to guide the second large language model to answer the analysis question.

7. The device according to claim 6, characterized in that The log screening module includes: A first screening submodule is configured to generate a second compressed log based on log lines of the target log type in the log to be analyzed; The second filtering submodule is configured to input the second compressed log into a third language model, obtain a first log line output by the third language model under the guidance of a third prompt word, and generate a first compressed log; wherein the first log line is a log line summarizing each log line in the second compressed log; and the third prompt word is used to guide the third language model to summarize each log line in the second compressed log.

8. The device according to claim 6, characterized in that The log screening module includes: A third screening submodule, configured to generate a third compressed log based on log lines of the target log type in the log to be analyzed; a fourth filtering submodule, configured to obtain log lines in the third compressed log whose lengths are greater than a preset length threshold, and generate a fourth compressed log; and obtain log lines in the third compressed log whose lengths are not greater than the preset length threshold, and generate a fifth compressed log; a fifth filtering submodule, configured to input the fourth compressed log into a fourth language model, obtain a second log line output by the fourth language model under the guidance of a fourth prompt word, and generate a sixth compressed log; wherein the second log line is a log line summarizing each log line in the fourth compressed log; and the fourth prompt word is used to guide the fourth language model to summarize each log line in the fourth compressed log; a sixth screening submodule, configured to generate a first compressed log based on the fifth compressed log and the sixth compressed log; The log screening module includes: a seventh screening submodule, configured to generate a seventh compressed log according to log lines of the target log type in the log to be analyzed; an eighth screening submodule, configured to obtain log lines in the seventh compressed log that meet a preset log condition and generate a first compressed log; wherein the preset log condition is used to determine logs that meet the log format requirements of the second largest language model for input logs; The eighth screening submodule comprises: a first screening unit, configured to obtain log lines that meet a preset log condition in the seventh compressed log and generate an eighth compressed log; a second screening unit configured to obtain, from the eighth compressed log, log lines whose lengths are greater than the preset length threshold, and generate a ninth compressed log; and obtain, from the eighth compressed log, log lines whose lengths are not greater than the preset length threshold, and generate a tenth compressed log; a third screening unit configured to input the ninth compressed log into a fifth language model, obtain a third log line output by the fifth language model under the guidance of a fifth prompt word, and generate an eleventh compressed log; wherein the third log line is a log line summarizing each log line in the ninth compressed log; and the fifth prompt word is used to guide the fifth language model to summarize each log line in the ninth compressed log; The fourth screening unit is configured to generate a first compressed log according to the tenth compressed log and the eleventh compressed log.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 5 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images