Log processing method, electronic device, and computer storage medium

Abstract

The present disclosure provides a log processing method, which comprises: determining a normal log control flow graph based on a historical log stream, wherein the historical log stream is generated under the condition that the system is running normally; determining a current log control flow graph based on a current log stream; determining an initial destination node of log printing in the current log control flow graph according to the normal log control flow graph and the current log control flow graph; and filtering the initial destination node of log printing according to a preset filtering algorithm to determine a final destination node of log printing. The abnormality is automatically discovered and the key log information that can be used to infer the root cause of the fault is determined, so that the automatic log enhancement can be realized without manual prior knowledge, the log quality is improved, and the efficiency and accuracy of fault diagnosis are improved. The present disclosure also provides an electronic device and a computer storage medium.

Description

Technical Field

[0001] This invention relates to the field of log technology, specifically to a log processing method, an electronic device, and a computer storage medium. Background Technology

[0002] With the development of Artificial Intelligence (AI), Artificial Intelligence for IT Operations (AIOps) was first proposed by Gartner in 2016. AIOps involves using machine learning algorithms to analyze large-scale data from various operation and maintenance tools and devices to automatically discover and respond to system problems in real time, thereby improving IT operation and maintenance capabilities and automation levels. Under the AIOps trend, automated and intelligent fault diagnosis, with system log data analysis at its core, has become an important component and development trend of distributed software system fault diagnosis technology.

[0003] Currently, although system logs are widely used in fault diagnosis technology, the logs written by developers are designed for humans, not machines, to automatically detect anomalies and diagnose faults. This results in the loss of much crucial log information that could be used to infer the root cause of faults. This characteristic of logs severely impacts the performance of automated and intelligent fault diagnosis technologies. While log enhancement techniques exist to improve log quality, such as adding extra variables or adding log print points at potential error locations, these techniques aim to enhance developers' understanding of the logs, not for automated and intelligent fault diagnosis. Furthermore, these techniques require significant prior human knowledge. Therefore, there is an urgent need for an automated log enhancement method that does not require prior human knowledge. Summary of the Invention

[0004] This disclosure addresses the aforementioned deficiencies in the prior art by providing a log processing method, an electronic device, and a computer storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a log processing method, the method comprising:

[0006] The normal log control flow graph is determined based on the historical log stream, wherein the historical log stream is generated under normal system operation.

[0007] Determine the current log control flow graph based on the current log flow;

[0008] Based on the normal log control flow graph and the current log control flow graph, determine the initial destination node for log printing in the current log control flow graph;

[0009] The initial destination node for log printing is filtered according to a preset filtering algorithm to determine the final destination node for log printing.

[0010] In some embodiments, the step of determining the log control flow graph based on the log flow includes:

[0011] Each log item in the log stream is converted into a log template;

[0012] Determine the edge weight between two log templates in each log template pair, wherein the log template pair includes two log templates corresponding to the same request;

[0013] Each log template is used as a node, and a transition edge between two log templates is constructed according to the edge weight to obtain the log control flow graph.

[0014] In some embodiments, determining the initial destination node for log printing in the current log control flow graph based on the normal log control flow graph and the current log control flow graph includes:

[0015] Match the normal log control flow graph and the current log control flow graph according to nodes;

[0016] The initial destination node for log printing is determined based on the node that matches the anomaly in the current log control flow graph.

[0017] In some embodiments, matching the normal log control flow graph and the current log control flow graph by node includes:

[0018] The nodes in the normal log control flow graph are matched pairwise with the nodes in the current log control flow graph to determine matching node groups. Each matching node group includes a normal node pair in the normal log control flow graph and a current node pair in the current log control flow graph. The normal node pair in each matching node group is matched with the current node pair.

[0019] Compare the edge weights between the normal node pairs in each of the matching node groups with the edge weights between the current node pairs;

[0020] When the edge weight between the current node pairs is greater than the edge weight between the normal node pairs, the current node pairs in the current matching node group are determined as the nodes with abnormal matching.

[0021] In some embodiments, determining the initial destination node for log printing based on nodes matching anomalies in the current log control flow graph includes:

[0022] If a node in the current log control flow graph does not match any node in the normal log control flow graph, then the node in the current log control flow graph is determined as the initial destination node for log printing.

[0023] In some embodiments, determining the initial destination node for log printing based on nodes matching anomalies in the current log control flow graph includes:

[0024] If the parent node in the current log control flow graph matches the parent node in the normal log control flow graph, and none of the child nodes after the parent node in the current log control flow graph match the child nodes after the parent node in the normal log control flow graph, then the parent node and its child nodes in the current log control flow graph are determined as the initial destination node for log printing.

[0025] In some embodiments, converting each log item in the log stream into a log template includes:

[0026] In response to receiving a conversion instruction, the variables in each of the log entries are replaced with placeholders, and the log template is generated based on the replaced log entries.

[0027] In some embodiments, determining the edge weight between any two log templates corresponding to the same request includes:

[0028] Determine the generation time intervals for the two log templates;

[0029] The edge weight between the two log templates is determined based on the maximum value among the generation time intervals.

[0030] In some embodiments, filtering the initial destination node of the log printing according to a preset filtering algorithm to determine the final destination node of the log printing includes:

[0031] Based on the current system fault information and the initial destination node of the log printing, construct a composite matrix;

[0032] The final destination node for log printing is determined using the preset filtering algorithm and the composite matrix.

[0033] In some embodiments, constructing the composite matrix based on the current system fault information and the initial destination node of the log printing includes:

[0034] Based on the current system fault information, construct a label vector F = {f1, f2, f3, ... f2}. m}, where m represents the total number of system faults, f m This indicates the m-th system failure;

[0035] Based on the initial destination node of the log printing, construct a vector T = t1, t2, t3, ... t n}, where n represents the total number of initial destination nodes for the log printing, and t n This indicates the initial destination node for the nth log entry.

[0036] Construct an association matrix M based on F and T, wherein the elements in M ​​are...

[0037] Construct the composite matrix D based on F and M, where,

[0038] Secondly, this disclosure provides an electronic device, comprising:

[0039] One or more processors;

[0040] A storage device on which one or more programs are stored;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement the log processing method as described above.

[0042] Thirdly, this disclosure provides a computer storage medium having a computer program stored thereon, wherein the program, when executed, implements the log processing method as described above.

[0043] By determining the normal log control flow graph based on the historical log stream generated under normal system operation, and determining the current log control flow graph based on the current log stream, the initial destination node for log printing in the current log control flow graph is determined according to the normal log control flow graph and the current log control flow graph. The initial destination node for log printing is filtered according to a preset filtering algorithm to determine the final destination node for log printing. This automatically detects anomalies and identifies key log information that can be used to infer the root cause of faults. Automated log enhancement can be achieved without human prior knowledge, improving log quality and helping to improve the efficiency and accuracy of fault diagnosis. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the log processing method provided in the embodiments of this disclosure. Figure 1 ;

[0045] Figure 2 This is a flowchart illustrating the log processing method provided in the embodiments of this disclosure. Figure 2 ;

[0046] Figure 3 This is a flowchart illustrating the log processing method provided in the embodiments of this disclosure. Figure 3 ;

[0047] Figure 4 This is a flowchart illustrating the log processing method provided in the embodiments of this disclosure. Figure 4 ;

[0048] Figure 5 This is a flowchart illustrating the log processing method provided in the embodiments of this disclosure. Figure 5 ;

[0049] Figure 6 This is a flowchart illustrating the log processing method provided in the embodiments of this disclosure. Figure 6 . Detailed Implementation

[0050] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0051] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0052] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the said feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0053] The embodiments described herein can be described with reference to plan views and / or cross-sectional views using the ideal schematic diagrams of this disclosure. Therefore, the example illustrations can be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to those shown in the drawings, but include modifications to configurations formed based on manufacturing processes. Therefore, the areas illustrated in the drawings are schematic in nature, and the shapes of the areas shown in the figures illustrate specific shapes of areas of an element, but are not intended to be limiting.

[0054] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0055] In existing technologies, logs written by developers are designed for human readability, not for machines to automatically detect anomalies and diagnose root causes. This leads to the loss of much crucial log information that could be used to infer the root cause of a fault. This characteristic of logs severely impacts the performance of automated and intelligent fault diagnosis technologies. Therefore, this disclosure proposes that anomalies can be discovered and root causes diagnosed by comparing the currently generated logs with logs generated during normal system operation. Specifically, logs themselves can represent certain information, and different logs are related to each other. Therefore, a log control flow graph can be generated based on the log flow. By directly comparing the current log control flow graph with a normal log control flow graph, abnormal nodes in the normal log control flow graph can be identified. These abnormal nodes are then filtered to obtain the nodes that ultimately need to be logged.

[0056] Correspondingly, such as Figure 1 As shown in the figure, this disclosure provides a log processing method, which may include the following steps:

[0057] In step S11, a normal log control flow graph is determined based on the historical log stream, wherein the historical log stream is generated under normal system operation.

[0058] In step S12, the current log control flow graph is determined based on the current log flow;

[0059] In step S13, the initial destination node for log printing in the current log control flow graph is determined based on the normal log control flow graph and the current log control flow graph.

[0060] In step S14, the initial destination node of the log printing is filtered according to a preset filtering algorithm to determine the final destination node of the log printing.

[0061] The log control flow graph includes edges between nodes. The initial destination node for log printing refers to the node initially identified as needing log printing, while the final destination node refers to the node ultimately identified as needing log printing. A preset filtering algorithm can be used to select the final destination node from the initial destination nodes. The final destination node contains the key log information that can be used to infer the root cause of the fault.

[0062] As can be seen from steps S11-S14 above, the log processing method provided in this embodiment determines a normal log control flow graph based on the historical log stream generated under normal system operation, determines a current log control flow graph based on the current log stream, determines the initial destination node for log printing in the current log control flow graph according to the normal log control flow graph and the current log control flow graph, filters the initial destination node for log printing according to a preset filtering algorithm to determine the final destination node for log printing, automatically detects anomalies and determines key log information that can be used to infer the root cause of the fault, achieves automated log enhancement without the need for manual prior knowledge, improves log quality, and helps improve the efficiency and accuracy of fault diagnosis.

[0063] Because it can automatically detect anomalies and identify key log information that can be used to infer the root cause of failures, the log processing method provided in this disclosure is also well-suited for DevOps technology (a combination of Development and Operations), a culture, movement, or practice that emphasizes communication and collaboration between software developers (Dev) and IT operations personnel (Ops) to make building, testing, and releasing software faster, more frequent, and more reliable by automating the processes of software delivery and architecture changes.

[0064] As mentioned earlier, logs themselves can represent some information, and different logs also have certain relationships. Each log item in the log stream can be transformed into a log template in turn, and the log templates can be used as nodes in the log control flow graph. There is a transition relationship between two log templates (or log items) from the same request (service request or method call request). Transition edges can be constructed between these two log templates, thus obtaining a log control flow graph that includes nodes and edges between certain nodes.

[0065] Accordingly, in some embodiments, such as Figure 2 As shown, the steps for determining the log control flow graph based on the log flow may include:

[0066] In step S21, each log item in the log stream is converted into a log template;

[0067] In step S22, the edge weight between two log templates in each log template pair is determined, wherein the log template pair includes two log templates corresponding to the same request;

[0068] In step S23, each log template is used as a node, and a transition edge between two log templates is constructed according to the edge weight to obtain the log control flow graph.

[0069] In this log template pair, the two log templates corresponding to the same request refer to two log templates from the same service request or method call request. For these two log templates, the edge weights need to be determined first so that the transition edge between the two log templates can be constructed later based on the edge weights.

[0070] It should be noted that whenever a new log entry is retrieved from the currently generated log stream, the current log control flow graph can be updated based on that new log entry. Furthermore, existing or trained fault diagnosis models can be used in real time to diagnose whether a fault has occurred in the current system. To save resources, step S13 can be executed only after a system fault has been detected. Of course, to more quickly detect anomalies and diagnose faults, it can also be executed automatically after each update of the current log control flow graph.

[0071] The log control flow graph includes nodes and edges between certain nodes. Abnormal nodes in the current log control flow graph can be discovered by matching the normal log control flow graph with the current log control flow graph according to nodes.

[0072] Accordingly, in some embodiments, such as Figure 3 As shown, determining the initial destination node for log printing in the current log control flow graph (i.e., step S13) based on the normal log control flow graph and the current log control flow graph may include the following steps:

[0073] In step S131, the normal log control flow graph and the current log control flow graph are matched according to nodes;

[0074] In step S132, the initial destination node for log printing is determined based on the nodes that match the anomalies in the current log control flow graph.

[0075] Among them, a node with an abnormal match refers to a node that exists in the current log control flow graph but cannot be matched with a node in the normal log control flow graph, or a node pair that, although both nodes can be matched with two nodes in the normal log control flow graph, cannot be matched with the edge between two nodes in the normal log control flow graph, etc.

[0076] Accordingly, in some embodiments, such as Figure 4 As shown, matching the normal log control flow graph and the current log control flow graph according to nodes (i.e., step S131) ​​may include the following steps:

[0077] In step S1311, the nodes in the normal log control flow graph are matched with the nodes in the current log control flow graph to determine matching node groups. Each matching node group includes a normal node pair in the normal log control flow graph and a current node pair in the current log control flow graph. The normal node pair in each matching node group is matched with the current node pair.

[0078] In step S1312, the edge weights between the normal node pairs in each of the matching node groups are compared with the edge weights between the current node pairs;

[0079] In step S1313, when the edge weight between the current node pairs is greater than the edge weight between the normal node pairs, the current node pairs in the current matching node group are determined as the nodes with abnormal matching.

[0080] After matching the normal log control flow graph and the current log control flow graph in pairs, if it is found that there are two nodes (i.e., node pairs) in the current log control flow graph that meet the following conditions: both of these nodes can be matched with nodes in the normal log control flow graph, and the edge weight between these two nodes is greater than the edge weight between two nodes in the normal log control flow graph that match these two nodes, then these two nodes are considered mismatched nodes, and all node pairs that meet the above conditions are considered mismatched nodes.

[0081] In this embodiment, the two nodes with an edge in the log control flow graph have a transition relationship, meaning they originate from the same request (service request or method call request). The edge weight is determined based on the generation time difference between the two nodes. When the edge weight between these two nodes is greater than the edge weight between two matching nodes in the normal log control flow graph, it indicates that the time interval between these two nodes is higher than normal, resulting in a latency anomaly. Both nodes can be marked with an anomaly flag, and the type of the anomaly flag is latency anomaly. These node pairs with latency anomaly flags can be determined as the initial destination nodes for log printing.

[0082] Furthermore, when there is a node in the current log control flow graph that does not match any node in the normal log control flow graph, it indicates that the appearance of such a node is unreasonable and a redundancy anomaly has occurred. Accordingly, in some embodiments, determining the initial destination node for log printing based on the node with an anomaly in the current log control flow graph (i.e., step S132) may include the following steps: when a node in the current log control flow graph does not match any node in the normal log control flow graph, determine the node in the current log control flow graph as the initial destination node for log printing.

[0083] After matching the normal log control flow graph and the current log control flow graph by node, if a node in the current log control flow graph is found that does not match any node in the normal log control flow graph, it indicates that this node is redundant and a redundancy anomaly has occurred. These nodes can be marked with an anomaly flag, and the anomaly flag type is redundancy anomaly. These nodes with redundancy anomaly flags can be determined as the initial destination nodes for log printing.

[0084] Furthermore, if a parent node in the current log control flow graph matches a parent node in the normal log control flow graph, and none of the child nodes following that parent node match the child nodes of that parent node in the normal log control flow graph, it indicates that the appearance of such a parent node and its subtree is also unreasonable, and a sequence anomaly has occurred.

[0085] Accordingly, in some embodiments, determining the initial destination node for log printing based on the nodes that match the abnormal nodes in the current log control flow graph (i.e., step S132) may include the following steps: when the parent node in the current log control flow graph matches the parent node in the normal log control flow graph, and none of the child nodes after the parent node in the current log control flow graph match the child nodes after the parent node in the normal log control flow graph, the parent node and its child nodes in the current log control flow graph are determined as the initial destination node for log printing.

[0086] After matching the normal log control flow graph and the current log control flow graph by node, if a parent node in the current log control flow graph is found to meet the following conditions: this parent node matches a parent node in the normal log control flow graph, and none of the child nodes (i.e., log entries) after this parent node match any child node after a parent node in the normal log control flow graph, it indicates that the appearance of this parent node and its child nodes is unreasonable, and a sequence anomaly has occurred. This parent node and its smallest subtree can be marked with an anomaly flag, and the anomaly flag type is sequence anomaly. These parent nodes with sequence anomaly flags and their smallest subtrees can be determined as the initial destination nodes for log printing.

[0087] Of course, after marking the identified nodes with anomaly flags, the initial destination node for log printing can be uniformly identified based on the type of its anomaly flag: node pairs with delay anomaly flags, nodes with redundancy anomaly flags, and parent nodes and their smallest subtrees with sequence anomaly flags can all be determined as the initial destination nodes for log printing.

[0088] Log template mining algorithms can be used to convert each log item in a log stream into a log template. A log template is a log type abstraction that uses constant portions of the log as identifiers. Accordingly, in some embodiments, converting each log item in the log stream into a log template (i.e., step S21) may include the following steps: in response to receiving a conversion instruction, replacing variables in each log item with placeholders, and generating the log template based on the replaced log items.

[0089] By retaining the constant portion of the log entry and marking the variable portion with placeholders, a log entry can be converted into a log template.

[0090] In the log control flow graph, nodes are log templates. A transition edge is constructed between two log templates corresponding to the same request. The weight of the transition edge can be determined based on the maximum time interval between the two log templates. Correspondingly, in some embodiments, such as... Figure 5 As shown, determining the edge weight between any two log templates corresponding to the same request (i.e., step S22) may include the following steps:

[0091] In step S221, the generation time intervals of the two log templates are determined;

[0092] In step S222, the edge weight between the two log templates is determined based on the maximum value among the generation time intervals.

[0093] The generation time interval is the time difference between the generation times of two log entries used to convert the two log templates. The edge weight of the transition edge between the two log templates is determined according to the maximum generation time interval between the two log templates.

[0094] After determining the initial destination node for log printing, it is necessary to filter the initial destination node to obtain a more accurate final destination node for log printing, which is the key log information that can be used to infer the root cause of the fault. Accordingly, in some embodiments, such as Figure 6 As shown, the step of filtering the initial destination node of the log printing according to the preset filtering algorithm to determine the final destination node of the log printing (i.e., step S14) may include the following steps:

[0095] In step S141, a composite matrix is ​​constructed based on the current system fault information and the initial destination node of the log printing;

[0096] In step S142, the final destination node for log printing is determined using the preset filtering algorithm and the composite matrix.

[0097] The composite matrix refers to a matrix that combines system fault information and the correlation between system fault information and the initial destination node of log printing.

[0098] Accordingly, in some embodiments, constructing the composite matrix based on the current system fault information and the initial destination node of the log printing (i.e., step S141) may include the following steps:

[0099] Based on the current system fault information, construct a label vector F = {f1, f2, f3, ... f2}. m}, where m represents the total number of system faults, f m This indicates the m-th system failure;

[0100] Based on the initial destination node printed in the log, construct a vector T = {t1, t2, t3, ... t}. n}, where n represents the total number of initial destination nodes for the log printing, and t n This indicates the initial destination node for the nth log entry.

[0101] Construct an association matrix M based on F and T, wherein the elements in M ​​are...

[0102] Construct the composite matrix D based on F and M, where,

[0103] The correlation matrix M represents the association between system fault information and the initial destination node for log printing. When the initial destination node for the j-th log printing is caused by the i-th system fault, a ij The value is 1 if it is not 0 otherwise.

[0104] D is a composite matrix based on attribute-value patterns, where the association matrix M, also known as Abnormal Pattern Sets (APS), represents attributes, while the system fault label vector F represents values. After constructing the composite matrix, the matrix can be input into a preset filtering algorithm, as shown below:

[0105]

[0106] The relevant notation used in the above algorithm is as follows:

[0107] Let Gain(A) be the sum of the values ​​of Gain and Gain given a condition A (such as a selected anomaly flag t). x Under the condition Info(D), the information entropy Info(D) and the conditional entropy Info(D) are... A The difference between (D) and Gain(A) is equal to Info(D) - Info. A(D). Gain(A) can be called the information gain under given condition A.

[0108] According to the definition of a composite matrix D, each element a in D ix The value can only be 0 or 1. Let m0 be the number of elements with a value of 0 and m1 be the number of elements with a value of 1. Since there are m system faults, m0 + m1 = m. Then the information entropy Info(D) and the conditional entropy Info are... A (D) can be calculated using the following formula:

[0109]

[0110]

[0111] As can be seen, the above-mentioned preset filtering algorithm has multiple iterations. In each iteration, the composite matrix D is calculated for each attribute a. ij Gain(a) under the condition ij ), from each Gain(a ij The highest value and the initial destination node for logging the corresponding highest value are determined in the process, and this is called t. best According to t, which has the highest information gain best Divide the current matrix into two submatrices D x and D y Then the algorithm proceeds to the next iteration. After the iteration terminates, the preset filtering algorithm outputs a filtered abnormal pattern set (FAPS) matrix, and the t corresponding to each element in the FAPS is the t with the highest information gain selected in each iteration. best These t best This constitutes the final set of destination nodes for log printing.

[0112] Furthermore, embodiments of this disclosure also provide an electronic device, including:

[0113] One or more processors;

[0114] A storage device on which one or more programs are stored;

[0115] When the one or more programs are executed by the one or more processors, the one or more processors implement the log processing method as described above.

[0116] Furthermore, this disclosure also provides a computer storage medium storing a computer program thereon, wherein the program, when executed, implements the log processing method as described above.

[0117] It will be understood by those skilled in the art that all or some of the steps in the methods disclosed above, and the functional modules / units in the apparatus, can be implemented as software, firmware, hardware, and suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0118] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A log processing method, characterized in that, The method includes: The normal log control flow graph is determined based on the historical log stream, wherein the historical log stream is generated under normal system operation. Determine the current log control flow graph based on the current log flow; Based on the normal log control flow graph and the current log control flow graph, determine the initial destination node for log printing in the current log control flow graph; The initial destination node for log printing is filtered according to a preset filtering algorithm to determine the final destination node for log printing. The step of filtering the initial destination node of the log printing according to a preset filtering algorithm to determine the final destination node of the log printing includes: Based on the current system fault information and the initial destination node of the log printing, a composite matrix is ​​constructed. The composite matrix is ​​a matrix that combines the system fault information and the correlation between the system fault information and the initial destination node of the log printing. The final destination node for log printing is determined using the preset filtering algorithm and the composite matrix.

2. The method according to claim 1, characterized in that, The steps to determine the log control flow graph based on log flow include: Each log item in the log stream is converted into a log template; For each pair of log templates corresponding to the same request, the edge weight between the two log templates is determined, and each log template is treated as a node. The edge weight is determined based on the difference in generation time between the two nodes. Based on the edge weights, construct the transition edges between the two corresponding log templates to obtain the log control flow graph.

3. The method according to claim 2, characterized in that, The step of determining the initial destination node for log printing in the current log control flow graph based on the normal log control flow graph and the current log control flow graph includes: Match the normal log control flow graph and the current log control flow graph according to nodes; The initial destination node for log printing is determined based on the node that matches the anomaly in the current log control flow graph.

4. The method according to claim 3, characterized in that, The step of matching the normal log control flow graph and the current log control flow graph according to nodes includes: The nodes in the normal log control flow graph are matched pairwise with the nodes in the current log control flow graph to determine matching node groups. Each matching node group includes a normal node pair in the normal log control flow graph and a current node pair in the current log control flow graph. The normal node pair in each matching node group is matched with the current node pair. Compare the edge weights between the normal node pairs in each of the matching node groups with the edge weights between the current node pairs; When the edge weight between the current node pairs is greater than the edge weight between the normal node pairs, the current node pairs in the current matching node group are determined as the nodes with abnormal matching.

5. The method according to claim 3, characterized in that, The step of determining the initial destination node for log printing based on the nodes that match the anomalies in the current log control flow graph includes: If a node in the current log control flow graph does not match any node in the normal log control flow graph, then the node in the current log control flow graph is determined as the initial destination node for log printing.

6. The method according to claim 3, characterized in that, The step of determining the initial destination node for log printing based on the nodes that match the anomalies in the current log control flow graph includes: If the parent node in the current log control flow graph matches the parent node in the normal log control flow graph, and none of the child nodes after the parent node in the current log control flow graph match the child nodes after the parent node in the normal log control flow graph, then the parent node and its child nodes in the current log control flow graph are determined as the initial destination node for log printing.

7. The method according to claim 2, characterized in that, The step of converting each log item in the log stream into a log template includes: In response to receiving a conversion instruction, the variables in each of the log entries are replaced with placeholders, and the log template is generated based on the replaced log entries.

8. The method according to claim 2, characterized in that, The step of determining the edge weight between two log templates corresponding to the same request for each pair includes: Determine the generation time intervals for the two log templates; The edge weight between the two log templates is determined based on the maximum value among the generation time intervals.

9. The method according to any one of claims 1-8, characterized in that, The step of constructing a composite matrix based on the current system fault information and the initial destination node of the log printing includes: Construct a label vector F based on the current system fault information. Where m represents the total number of system faults, This indicates the m-th system failure; Construct a vector based on the initial destination node printed in the log. Where n represents the total number of initial destination nodes for the log printing. This indicates the initial destination node for the nth log entry. Construct an association matrix M based on F and T, wherein the elements in M ​​are... , ; Construct the composite matrix D based on F and M, where D .

10. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the log processing method as described in any one of claims 1-9.

11. A computer storage medium having a computer program stored thereon, wherein, When the program is executed, it implements the log processing method as described in any one of claims 1-9.

Images