Log information identification method and device, equipment, medium and product

Abstract

The invention discloses a log information identification method and device, equipment, a medium and a product, and the method comprises the steps: obtaining a to-be-processed log file and a pre-constructed key information search tree, and constructing the key information search tree based on preset substring format information; based on a node substring structure of each node in the key information search tree, carrying out key information identification on a substring of the to-be-processed log file to obtain key information; and updating nodes in the key information search tree based on the key information, and storing the key information in the log objects until an identification ending condition is met, thereby obtaining all the log objects of the to-be-processed log file. The key information search tree corresponding to the to-be-processed log file is constructed through the preset substring format information, the key information in the to-be-processed log file is matched in a substring mode through the key information search tree, character matching one by one is not needed any more, the matching speed is increased, a lexical analysis program of a third party does not need to be relied on, and the convenience of the method is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a log information identification method, device, equipment, medium and product. Background Art

[0002] When a database fails during operation, operation and maintenance personnel usually use the log information in the log file to analyze and troubleshoot the cause. As the database runs longer, more and more log information will be recorded. If the log information is obtained only by manually viewing the log file, the efficiency will be very low and it will be prone to errors.

[0003] The prior art uses a log information recognition program to identify each log message in a log file, and then identifies each log message as key information representing different meanings according to a specific format, and stores it in a database table in a structured manner, so that all the information of the log message can be queried by fuzzy querying the key information of the log message. Usually in the log information recognition program, the format of defining key information is regular expressions or lexical rules, plus a small amount of grammatical rules. The recognition process is to first divide the entire log message into a word list separated by spaces, and then match it with the defined key information format in turn. If it matches, the semantics of the word is recognized as the key information defined in the format.

[0004] However, this method has two disadvantages: First, the matching method is performed character by character, and when there are a lot of logs in the log file, the matching speed is not fast enough. Second, it needs to rely on a third-party lexical analysis program and learn lexical grammar rules, which is not convenient enough. Summary of the invention

[0005] The present invention provides a log information identification method, device, equipment, medium and product, which realizes rapid identification of key log information by substring identification.

[0006] According to a first aspect of the present invention, a log information identification method is provided, comprising:

[0007] Obtaining a log file to be processed and a pre-constructed key information search tree, wherein the key information search tree is constructed based on preset substring format information;

[0008] Based on the node substring structure of each node in the key information search tree, key information is identified on the substring of the log file to be processed to obtain key information;

[0009] The nodes in the key information search tree are updated based on the key information, and the key information is stored in a log object until an identification end condition is met, thereby obtaining all the log objects of the log file to be processed.

[0010] According to a second aspect of the present invention, there is provided a log information identification device, comprising:

[0011] A file acquisition module, used to acquire a log file to be processed and a pre-built key information search tree, wherein the key information search tree is built based on preset substring format information;

[0012] An information identification module, used to identify key information of the substring of the log file to be processed based on the node substring structure of each node in the key information search tree to obtain key information;

[0013] The information saving module is used to update the nodes in the key information search tree based on the key information, and save the key information in the log object until the identification end condition is met, thereby obtaining all the log objects of the log file to be processed.

[0014] According to a third aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log information identification method described in any embodiment of the present invention.

[0018] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the log information identification method described in any embodiment of the present invention when executed.

[0019] According to a fifth aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the log information identification method of any embodiment of the present invention is implemented.

[0020] The technical solution of the embodiment of the present invention obtains the log file to be processed and the pre-constructed key information search tree, and the key information search tree is constructed based on the preset substring format information; based on the node substring structure of each node in the key information search tree, the substring of the log file to be processed is identified for key information to obtain key information; based on the key information, the nodes in the key information search tree are updated, and the key information is saved in the log object until the identification end condition is met, and all log objects of the log file to be processed are obtained. The key information search tree corresponding to the log file to be processed is constructed by the preset substring format information, and the key information in the log file to be processed is matched in the form of a substring through the key information search tree, and no longer matches characters one by one, thereby improving the matching speed, and no longer relying on a third-party lexical analysis program, thereby improving the convenience of the method.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a log information identification method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 is a flow chart of constructing a search tree of a log information identification method provided in accordance with the first embodiment of the present invention;

[0025] Figure 3 is a flow chart of a log information identification method provided according to Embodiment 2 of the present invention;

[0026] Figure 4 is a search example flow chart of a log information identification method provided according to Embodiment 2 of the present invention;

[0027] Figure 5 is an example flow chart of a log information identification method provided according to Embodiment 2 of the present invention;

[0028] Figure 6 is a structural schematic diagram of a log information identification device provided according to Embodiment 3 of the present invention;

[0029] Figure 7It is a schematic diagram of the structure of an electronic device implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 A flow chart of a log information identification method is provided for the first embodiment of the present invention. This embodiment can be applied to extracting information from log files. The method can be executed by a log information identification device. The log information identification device can be implemented in the form of hardware and / or software. The log information identification device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0034] S110: Obtain a log file to be processed and a pre-constructed key information search tree, where the key information search tree is constructed based on preset substring format information.

[0035] In this embodiment, the log file to be processed can be understood as a log file that needs to be extracted. The key information search tree can be understood as a search form of a tree structure for identifying key information in the log file. The preset substring format information can be understood as a format for defining key information and describing the key information structure.

[0036] Specifically, the processor may obtain the log file to be processed, and search the storage medium for a pre-built key information search tree corresponding to the log file to be processed.

[0037] S120 , based on the node substring structure of each node in the key information search tree, perform key information identification on the substring of the log file to be processed to obtain key information.

[0038] In this embodiment, a node can be understood as each part of the tree structure that constitutes the key information search tree, for example, it can include a root node and a child node. A node substring structure can be understood as a structure for describing a string corresponding to each node. A substring can be understood as a continuous segment of characters in a string. Key information can be understood as information with semantics in a log file.

[0039] Specifically, the processor can read non-space log characters in the log file to be processed, record the starting recognition position as the current log position of the log character, and search in the key information search tree. By determining the node substring structure of the child nodes in the key information search tree, the end position of the divided characters can be determined, and based on the node substring structure, the key information of the log characters between the starting recognition position and the end position can be determined, as well as the type corresponding to the key information. The type of key information can be used to distinguish whether it is log content or the start mark of a new key information.

[0040] S130: updating the nodes in the key information search tree based on the key information, and saving the key information in the log object until the identification end condition is met, and obtaining all log objects of the log file to be processed.

[0041] In this embodiment, the log object can be understood as a component used to record key information of the same type, for example, log object 1 records the name of the program that generates the log, log object 2 records the time when the log is generated, and log object 3 records the level of the log generated, etc. The identification end condition can be understood as having read all the log files to be processed.

[0042] Specifically, since a log file to be processed may contain many pieces of key information, in order to form a log object for key information belonging to the same category, the processor may traverse the types of key information in turn to determine whether the currently traversed key information is already the beginning of the next category of key information. If so, that is, all nodes corresponding to all key information belonging to a log object have been identified in the key information search tree, then these nodes may be removed to update the nodes in the key information search tree, the obtained log object may be saved, and a new log object may be created to store all key information of the next category, and the determination of the type of the next key information may be continued; if not, this category of key information has not been completely stored in the current log object, then the key information may be stored in the current log object, and the determination of the type of the next key information may be continued, until the log file to be processed has been completely read and there is no unread key information, the identification end condition is met, and all log objects of the log file to be processed are obtained.

[0043] The technical solution of the embodiment of the present invention obtains the log file to be processed and the pre-constructed key information search tree, and the key information search tree is constructed based on the preset substring format information; based on the node substring structure of each node in the key information search tree, the substring of the log file to be processed is identified for key information to obtain key information; based on the key information, the nodes in the key information search tree are updated, and the key information is saved in the log object until the identification end condition is met, and all log objects of the log file to be processed are obtained. The key information search tree corresponding to the log file to be processed is constructed by the preset substring format information, and the key information in the log file to be processed is matched in the form of a substring through the key information search tree, and no longer matches characters one by one, thereby improving the matching speed, and no longer relying on a third-party lexical analysis program, thereby improving the convenience of the method.

[0044] As a first optional embodiment of the first embodiment, based on the above embodiment, the preset substring format information includes a string representation format and a substring structure description. Accordingly, the steps of constructing the key information search tree include:

[0045] a1. Obtain the log file template of the log file to be processed.

[0046] In this embodiment, the log file template can be understood as a template used to characterize how the log file is composed.

[0047] Specifically, the processor may obtain a log file template of a log file to be processed defined by a relevant developer, and the log files to be processed subsequently generated have the same format as the log file template.

[0048] b1. Categorize the key information contained in the log file template according to the string representation format to obtain a key information substring containing a substring type.

[0049] In this embodiment, the string representation format can be understood as a pre-set format for representing different strings. The substring type can be understood as a form for distinguishing different key information categories. The key information substring can be understood as key information containing a substring type.

[0050] It can be known that the structure of the log file template is: the database operation log file contains multiple log messages, each of which is separated by a line break. Each log message consists of the log level, the time when the log is generated, the name of the log generating program, the process number of the log generating process, the thread number of the log generating process, and the detailed log content.

[0051] Specifically, the processor may match the key information contained in the log file template according to the string representation format, classify the key information in different representation forms, and obtain the key information substring containing the substring type.

[0052] For example, the level, time, process number, thread number, and log content in the log file template can be classified into different key information, and the different key information can be classified according to the string representation format. For example, the string representation format can include four types: starting constant substring, ending constant substring, fixed-length non-constant substring, and variable-length non-constant substring.

[0053] Among them, the starting constant substring refers to a substring with a fixed length, a fixed value, and starting at the first position; the starting point of each item in the log information; the ending constant substring refers to a substring with a fixed length, a fixed value, and ending at the last position; the fixed-length non-fixed substring specifies a substring with a fixed length and a non-fixed value; the variable-length non-fixed substring refers to a substring with a non-fixed length and a non-fixed value.

[0054] Furthermore, taking a specific example, the string format of key information and the classification results of the corresponding string formats can be shown in the following table:

[0055] Table 1 Comparison table

[0056]

[0057]

[0058] It can be seen that each key information may include at least one substring type. For example, when the key information is a program name, it includes a start constant substring, an end constant substring, and a variable-length non-constant substring.

[0059] c1. Determine the key information linked list based on the substring structure description and each key information substring.

[0060] In this embodiment, the substring structure description can be understood as a general description format for defining a substring, for example, it stores a series of information such as the substring type, the constant value involved in the substring, the length count of the non-constant substring, the location of the log identification substring, and the key information category meaning. The key information linked list can be understood as a linked list composed of all substring structures in a key information.

[0061] Specifically, the processor may initialize a linked list composed of substring structures through a substring structure description, determine the adding method corresponding to the substring category through a substring category included in a key information, and append the substring to the linked list.

[0062] Further, based on the above embodiment, the steps of determining the key information linked list according to the substring structure description and each key information substring can be refined as follows:

[0063] c11. Initialize the key information linked list composed of substring structure descriptions.

[0064] In this embodiment, the key information linked list can be understood as a key information linked list defined by a substring structure description.

[0065] For example, a specific example is used to illustrate the substring structure description:

[0066] Table 2 Substring structure description

[0067]

[0068]

[0069] A substring structure description contains five fields, namely, the substring type type, the constant value involved in the substring, the length of the non-constant substring count, the direction location of the log identification substring, and the key information category meaning. The substring structure description is formed by defining the data type and value range.

[0070] c12. For each key information substring, if there is a starting constant substring in the substring type, the key information substring is divided into multiple first substrings by characters, the first substring description information of the first substring is changed, and each first substring is added to the key information linked list in sequence according to the first order.

[0071] In this embodiment, the starting constant substring can be understood as a substring used to represent the starting position in the key information substring, for example, it can be the symbol "[" when the key information is the program name in Table 1. The first substring can be understood as the result obtained after character segmentation. The first substring description information can be understood as information representing the substring structure of the first substring. The first order can be understood as the order of adding the first substring in the linked list.

[0072] Specifically, for each key information substring, the processor can determine the types of multiple substrings contained in the key information substring. If there is a starting constant substring in the substring type, the substring under the type is divided into multiple first substrings per character. Each first substring has a defined substring structure representation, and the first substring description information of the first substring can be changed through the substring structure representation. For example, referring to Table 2, the type field of the substring takes the value of TYPE_CONST, and the direction field of the log identification substring takes the value of LOCATION_FIRS. And each first substring is added to the key information linked list in the first order, for example, it can be appended to the key information linked list from left to right.

[0073] c13. If there is an end constant substring in the substring type, the key information substring is divided into multiple second substrings by characters, the second substring description information of the second substring is changed, and each second substring is added to the key information linked list in sequence according to the second order.

[0074] In this embodiment, the end constant substring can be understood as a substring in the key information substring used to represent the end position, for example, it can be the symbol "]" when the key information is the program name in Table 1. The second substring can be understood as the result obtained after character segmentation. The second substring description information can be understood as information representing the substring structure of the second substring. The second order can be understood as the order of adding the second substring in the linked list.

[0075] Specifically, if there is an end constant substring in the substring type of the key information substring, the processor can divide the key information under this type into multiple second substrings according to the character unit, and change the second substring description information of the second substring, for example, the type field of the substring takes the value of TYPE_CONST, and the direction location field of the log identification substring takes the value of LOCATION_LAST. The second substrings are added to the key information linked list in the second order, for example, the second order can be appended to the linked list from right to left.

[0076] c14. If there is a fixed-length non-constant substring in the substring type, the key information substring is divided into multiple substring structures in which the length field of the non-constant substring takes the value of 1, and each substring structure is added to the key information linked list.

[0077] In this embodiment, the fixed-length non-constant substring can be understood as a substring with a fixed length and a non-constant value. The substring structure can be understood as a structure used to define the non-constant substring after segmentation.

[0078] Specifically, if there is a fixed-length non-constant substring in the substring type of the key information substring, the processor can divide the key information substring into multiple substring structures whose length count field of the non-constant substring takes the value of 1, and add each substring structure to the key information linked list.

[0079] c15. If there is a variable-length non-constant substring in the substring type, the key information substring is represented by the substring structure and added to the key information linked list.

[0080] Specifically, if there is a variable-length non-constant substring in the substring type of the key information substring, the key information substring is represented by the above substring structure and added to the key information linked list.

[0081] c16. According to the last substring structure in the associated information linked list, determine the category value of the key information substring, and obtain the key information linked list filled with the key information substring.

[0082] In this embodiment, the category value can be understood as a value that characterizes the type of each key information linked list.

[0083] Specifically, the processor may set the value of the key information category meaning field to be the category value of the key information according to the last substring structure in the associated information linked list, and obtain the key information linked list filled with the key information substring.

[0084] d1. Generate a key information search tree based on the key information linked list.

[0085] Specifically, the processor may start with the child nodes of the root node and traverse the log file template in a depth-first manner until the leaf node ends. A path chain represents a key information, and a key information search tree corresponding to the key information chain list is obtained.

[0086] Further, based on the above embodiment, the step of generating a key information search tree according to the key information chain list can be refined as follows:

[0087] d11. According to the log file template, determine the node structure of all nodes in the initial search tree, and convert the key information linked list into a tree node linked list under the node structure, and use the initial search tree as the current search tree; among which, the node structure of the root node is empty, and the node structure of the child node is described by a substring structure.

[0088] In this embodiment, the initial search tree can be understood as the search tree after initialization. The node can be understood as all nodes constituting the initial search tree. The node structure can be understood as the substring structure corresponding to each node. The tree node linked list can be understood as the structure of converting the substring structure linked list into a tree structure linked list form. The root node can be understood as the topmost node in the initial search tree. The child node can be understood as all nodes belonging to the lower level of the root node, which can also include leaf nodes, i.e., nodes at the lowest level.

[0089] Specifically, the processor can determine the node structure of all nodes in the initial search tree based on the log file template, and convert the key information linked list into a tree node linked list under the node structure, and use the initial search tree as the current search tree; wherein the node structure of the root node is empty, and the node structure of the child node is described by a substring structure.

[0090] Exemplarily, the initial search tree is denoted as searchTree, and its root node is designated as root. The substring structure of root is empty. The node structure of the initial search tree is denoted as TreeNode, and the TreeNode structure contains a substring structure. The converted key information linked list is strDescTree: that is, the key information linked list is converted into a TreeNode structure linked list, denoted as strDescTree. The current search tree can be understood as the search tree currently being searched.

[0091] d12. Traverse the tree node linked list, use the tree node linked list that has not generated the search tree as the current linked list to be generated, and determine the linked list root node and linked list child nodes of the current linked list to be generated.

[0092] In this embodiment, the linked list root node can be understood as the topmost node in the currently generated linked list. The linked list child node can be understood as the node in the currently generated linked list that is under the root node.

[0093] Specifically, the processor may traverse the tree node linked list, select the tree node linked list in which the search tree has not been generated as the current linked list to be generated, and determine the linked list root node and linked list child nodes of the current linked list to be generated.

[0094] d13. If the root node of the current search tree has a tree child node, determine whether the tree child node and the linked list root node meet the equality condition. If the equality condition is met, determine whether the linked list root node has a linked list child node.

[0095] In this embodiment, the equality condition can be understood as a condition for matching the nodes of the search tree with the nodes in the linked list. For example, the equality condition of the substring structure of the two nodes may include: the substring type type is equal; the direction location of the log identification substring is equal; if the substring type type is TYPE_CONST, the constant value involved in the substring must be equal; if the substring type type is TYPE_DIGIT or TYPE_LETTER or TYPE_DIGIT_OR_LETTER, the length count of the non-constant substring must be equal.

[0096] Specifically, if the root node of the current search tree has a tree child node, the substring structure of the tree child node is compared with the substring structure of the linked list root node to determine whether the equality condition is met. If the equality condition is met, it is determined whether there is a linked list child node under the linked list root node.

[0097] d14. If the equality condition is not met or the tree child node does not exist, the linked list root node is added to the tree child node list of the tree root node, the current search tree is updated, and the tree node linked list is continued to be traversed.

[0098] In this embodiment, the tree child node list can be understood as a list for recording the child nodes included in the hierarchy below the tree root node.

[0099] Specifically, if the tree child node and the linked list root node do not meet the equality condition, or the tree child node does not exist, the processor can add the linked list root node to the tree child node list of the tree root node, obtain the updated current search tree, and continue to traverse the tree node linked list.

[0100] d15. If there is a linked list child node, the tree child node is used as the tree root node, the linked list child node is used as the linked list root node, and the judgment of whether there is a tree child node is continued with the tree root node.

[0101] Specifically, if there is a linked list child node, the processor can use the tree child node as the tree root node and the linked list child node as the linked list root node, and continue to determine whether there is a tree child node with the tree root node, that is, return to continue to execute step d13.

[0102] d16. If there is no linked list child node, continue to traverse the tree node linked list.

[0103] Specifically, if the linked list root node does not have a linked list child node, the tree node linked list continues to be traversed, and the next tree node linked list that has not generated a search tree is returned to be used as the step of the current linked list to be generated.

[0104] d17. When the traversal is completed, the current search tree is compressed to obtain the key information search tree.

[0105] Specifically, when the traversal is finished, the processor may traverse the current search tree in a depth-first manner to determine the nodes therein that can be merged or sorted, so as to perform the merging and sorting operations to obtain a compressed key information search tree.

[0106] Among them, the steps of compressing the current search tree to obtain the key information search tree can be refined as follows:

[0107] If there is only one child node for the parent node in the current search tree, and the substring structures of the parent node and the child node are equal, then the first substring structure of the child node is merged into the second substring structure of the parent node to obtain a merged node, and the child node is deleted to obtain an updated search tree; the nodes with the same depth in the updated search tree are used as nodes to be sorted, and the third substring structure of each node to be sorted is determined; each node to be sorted is sorted according to each third substring structure to obtain a key information search tree.

[0108] In this embodiment, the parent node can be understood as a node that has a subordinate relationship with the child node, and the child node is located at the next level of the parent node. The first substring structure can be understood as a substring structure representing a child node that can be merged. The second substring structure can be understood as a substring structure representing a parent node that can be merged. The merged node can be understood as the node obtained after the merge. The nodes to be sorted can be understood as all nodes with the same depth. The third substring structure can be understood as the substring structure of the nodes to be sorted.

[0109] Specifically, in the current search tree, if a parent node has only one child node, and the substring structure of the parent node is equal to the substring structure of the child node, then the first substring structure of the child node can be merged into the second substring structure of the parent node to obtain a merged node. After the merge, the child node is deleted, and the child node of the deleted child node is used as the new child node of the parent node, so that the depth of the branch where the parent node is located is reduced by one degree. The processor can update the updated search tree according to the merged updated search tree. The nodes with the same depth are used as nodes to be sorted, and the third substring structure of each node to be sorted is determined. The nodes to be sorted are sorted according to the type value and field value contained in each third substring structure to obtain a key information search tree.

[0110] Exemplarily, the merging may be performed in the following manner: the type of the substring type and the direction location of the log identifying the substring remain unchanged; if the type of the substring type is TYPE_CONST, the constant value involved in the substring is appended; if the type of the substring type is TYPE_DIGIT or TYPE_LETTER, or TYPE_DIGIT_OR_LETTER, the length count of the non-constant substring is accumulated.

[0111] Exemplarily, the sorting can be performed in the following manner: Step 1, compare the type field of the substring in the third substring structure. When the type field values ​​of the substrings are different, the smaller the value, the higher the priority. The priority is TYPE_CONST>TYPE_DIGIT>TYPE_LETTER>TYPE_DIGIT_OR_LETTER. If the type of the substrings is the same, proceed to step 2.

[0112] Step 2: If the substring type type starts with TYPE_CONST, compare the constant value field value involved in the substring. The data type of the value field value is a string. If the substring type type is TYPE_DIGIT or TYPE_LETTER or TYPE_DIGIT_OR_LETTER, compare the length count field value of the non-normal substring. The data type of the length count field value of the non-normal substring is an integer.

[0113] The first optional embodiment of the first embodiment of the present invention is to split the key information in the form of substrings by defining the string representation format in the preset substring format information, and convert it through the substring structure description. By forming the key information substrings into a key information linked list, a basis is provided for the subsequent formation of a key information search tree. The key information search tree is identified and generated in the form of a substring linked list, which improves the generation speed and saves memory. The current search tree is generated through the key information linked list, and the current search tree is compressed and sorted to generate a key information search tree. The depth of the key information search tree is compressed, which helps to identify key information faster and more accurately.

[0114] For example, in order to facilitate understanding of the steps of constructing the key information search tree in the present invention, a specific example is used for demonstration. Figure 2 A flow chart of a search tree construction of a log information identification method provided in the first embodiment of the present invention, such as Figure 2 As shown, the steps may include:

[0115] S201, determine whether there is a tree node list strDescTree (specific definition see the description in the previous text) for the next ungenerated search tree; if so, jump to step S202; if not, end;

[0116] S202, let the root node of the current search tree searchTree (for specific definition, see the description in the previous text) be recorded as parentNode, and the root node of the current linked list strDescTree to be generated be recorded as descNode;

[0117] S203, determine whether the root node parentNode of the current search tree has a child node; if so, jump to S204; if not, jump to step S206;

[0118] S204, get the child node of the tree root node parentNode, record it as tempNode;

[0119] S205, judging whether the tree child node tempNode of the tree root node is equal to the linked list root node descNode of the linked list to be generated, if they are equal, jumping to S207; if they are not equal, jumping to step S206;

[0120] S206, adding the linked list root node descNode of the linked list to be generated to the tree child node list of the tree root node parentNode of the current search tree, and jumping to step S201;

[0121] S207, determine whether the linked list root node descNode of the linked list to be generated has a linked list child node. If it exists, jump to step S208; if not, go to S201;

[0122] S208, record the tree child node tempNode of the tree root node as parentNode, record the linked list child node of the linked list root node descNode of the linked list to be generated as descNode, and jump to step S203.

[0123] Embodiment 2

[0124] Figure 3 This is a flow chart of a log information identification method provided in Embodiment 2 of the present invention. This embodiment is a refinement of the above embodiment. Figure 3 As shown, the method includes:

[0125] S301, obtaining a log file to be processed and a pre-built key information search tree.

[0126] S302: Read the non-space characters currently traversed in the log file to be processed, and use the current log position of the characters in the log file to be processed as the starting recognition position.

[0127] In this embodiment, the current log position can be understood as the position of non-space characters in the log file to be processed. The starting identification position can be understood as a starting position for determining a key information in the log file to be processed.

[0128] Specifically, the processor may traverse the log file to be processed by characters, read the non-space characters currently traversed in the log file to be processed, and use the current log position of the characters in the log file to be processed as the starting recognition position.

[0129] S303: Determine the termination position according to the current root node of the key information search tree, the number of current child nodes corresponding to the current child node, and the node substring structure.

[0130] In this embodiment, the current root node can be understood as the root node currently being searched. The current child node can be understood as the child node currently being searched. The current number of child nodes can be understood as the number of child nodes subordinate to the current root node. The node substring structure can be understood as the node substring structure. The end position can be understood as the end position used to characterize the key information in the log to be processed.

[0131] Specifically, the processor can determine whether the substring corresponding to the key information has been extracted based on whether the current number of child nodes of the current child node is 0. If so, the current position is used as the end position. If not, the processor continues to determine whether the substring corresponding to the log content has been read by checking whether the current root node has the next child node. If not, the processor can use the sum of the length of the line of log and the starting identification position as the end position. If so, the processor can continue to identify the log characters through the node substring structure.

[0132] S304: In the log file to be processed, the substrings in the interval corresponding to the start identification position and the end position are identified as key information based on the node substring structure, and the information category of the key information is determined.

[0133] In this embodiment, the information category can be understood as information representing the category to which the key information belongs.

[0134] Specifically, in the log file to be processed, the substrings in the intervals corresponding to the start identification position and the end position are identified as key information based on the node substring structure, the log characters are identified through the node substring structure, and the information category of the key information is determined.

[0135] For the sake of illustrative purposes, a specific example may be used to illustrate the present invention. Figure 4 A search example flow chart of a log information identification method provided in Embodiment 2 of the present invention, such as Figure 4 As shown, the end position can be determined by the following steps:

[0136] S401, read the next non-space log character;

[0137] S402, the root node of the key information search tree searchTree is recorded as parentNode, and the starting identification position is recorded as the current log position;

[0138] S403, determine whether the number of current child nodes of the current child node of the current root node parentNode is 0; if it is 0, jump to step S404, if not 0, jump to step S405;

[0139] S404, identifying the substring of the log information character string position interval [starting identification position, current position) as the key information of the key information category meaning in the substring structure strDesc information of the search tree root node parentNode, and ending;

[0140] S405, determine whether the search tree root node parentNode has a next child node; if not, jump to step S406; if so, jump to step S407;

[0141] S406, identifying the substring of the log information character string position interval [starting identification position, length of the log line) as key information of the log content category, and ending;

[0142] S407, get the next child node of the current root node parentNode of the search tree, record it as tempNode;

[0143] S408, judging whether the node substring structure strDesc of the next child node tempNode has been identified; if so, jumping to step S409; if not, jumping to step S410;

[0144] S409, record the next child node tempNode as the search tree root node parentNode, and jump to step S403;

[0145] S410, judging whether the log character is recognized by the node substring structure strDesc of tempNode, if the recognition fails, jumping to step S405, if the recognition passes, jumping to step S405.

[0146] Exemplarily, the steps of identifying log characters through the node substring structure strDesc may be:

[0147] Step 1, define the index variable findIndex, with an initial value of 0, which represents the serial number of the next recognized character in the substring structure strDesc. When the current substring structure strDesc is not recognized or the recognition is completed, the index findIndex is reset to 0. In addition, the judgment conditions for the end of recognition are supplemented: if the type of the substring of the substring structure strDesc is TYPE_CONST, the condition is findIndex == value string length; if the type of the substring of the substring structure strDesc is not TYPE_CONST and the length of the non-normal substring count>0, the condition is findIndex == count value; if the type of the substring of the substring structure strDesc is not TYPE_CONST and the length of the non-normal substring count<=0, the condition is to recognize until the first space log character encountered. Step 2, take the log character to be compared. If the direction of the log recognition substring location == LOCATION_FIRST, take the character at the next position. If the direction of the log recognition substring location == LOCATION_LAST, then first find the first space character and then take the character at the previous position. Step 3: Determine whether the substring structure strDesc can recognize the current log character. If the substring type type is TYPE_CONST, then read the log character of the length of the value string - 1, that is, the total length of the log string with comparison is equal to the length of the value string. If it can be recognized, findIndex accumulates the length of the value string. If the substring type type is TYPE_DIGIT, then determine whether the log character is a number; if the substring type type is TYPE_LETTER, then determine whether the log character is a letter; if the substring type type is TYPE_DIGIT_OR_LETTER, then determine whether the log character is a number or a letter; if it can be recognized, findIndex accumulates 1.

[0148] S305: Remove the nodes that meet the identified conditions in the key information search tree, and continue traversing until the traversal is completed.

[0149] In this embodiment, the identified condition can be understood as a condition for determining whether the search is completed.

[0150] Specifically, the processor can judge the node. If the node is neither the starting key information node nor the key information node such as the log content that can be identified multiple times, it will be removed and saved in the stack data structure. Each time a non-starting key information is identified, it will be traversed upward from the last identified leaf node to remove it. The process can be: record the last identified leaf node as tempNode. Determine whether the last identified leaf node tempNode is the only child node of its parent node; if not, delete the last identified leaf node tempNode from the child node list of the parent node. Then save the last identified leaf node tempNode into the stack; if so, assign tempNode to the parent node of the last identified leaf node, and repeat the above judgment operation. After removal, continue to traverse the log file to be processed until the traversal is completed.

[0151] S306: When the information category of the key information is log content, the text of the key information is saved in the current log object.

[0152] In this embodiment, the log content can be understood as a detailed description of key information.

[0153] Specifically, when the information category of the key information is log content, the processor may save the text of the key information into the current log object.

[0154] S307: When the information category of the key information is the starting key information, the current log object is saved as the previous log object, a new log object is created as the current log object and the starting key information is stored, and the nodes in the key information search tree are restored.

[0155] In this embodiment, the previous log object can be understood as a log object storing the previous key information.

[0156] Specifically, when it is determined through the meaning category in the node substring structure of the current parent node that the information category of the key information is the starting key information, the processor can save the current log object as the previous log object, create a new log object as the current log object and store the starting key information, that is, the key information is a new key information, then restore the removed node from the stack, pop the node, and insert the node into the first child node list of its parent node.

[0157] S308: until the identification end condition is met, all log objects of the log file to be processed are obtained.

[0158] Specifically, when all the log files to be processed are read, the identification end condition is met, and all the obtained log objects are used as the reading results of the log files to be processed.

[0159] The technical solution of the embodiment of the present invention reads non-space characters in the log file to be processed as the starting identification position, combines the root node and child nodes of the key information search tree, determines the end position, identifies the substring in the log file to be processed through the node substring structure, and determines the information category of the corresponding key information, so as to distinguish whether the read log content belongs to the same log object as the content read last time through the information category, and then stores different key information in different log objects, thereby realizing the distinction of key information and the preservation of different log objects. By taking the substring within the starting identification position and the end position interval as the key information, the key information in the log file to be processed is matched in the form of a substring, and no character matching is performed one by one, thereby improving the matching speed. There is no need to rely on a third-party lexical analysis program, which improves the convenience of the method, and during maintenance, it is only necessary to add a key information format according to the preset substring format, thereby improving the convenience of maintenance.

[0160] For example, in order to facilitate understanding of the application process of matching log information through the key information search tree in the present invention, a specific example is used for demonstration. Figure 5 This is an example flow chart of a log information identification method provided in Embodiment 2 of the present invention, such as Figure 5 As shown, the steps may be:

[0161] S501, determine whether the log file to be processed has been read, if so, end, if not, jump to step S502;

[0162] S502, read a line of log;

[0163] S503, identifying a key information through a key information search tree;

[0164] S504, determine whether the category is log content, if so, jump to step S505; if not, jump to step S506;

[0165] S505, saving the identified key information text to the current log object, and returning to step S501;

[0166] S506, determine whether the information category is the starting key information, if so, jump to step S509, if not, jump to step S507;

[0167] S507, saving the identified key information text to the current log object;

[0168] S508, removing nodes that meet the identified conditions from the key information search tree;

[0169] S509, save the current log object;

[0170] S510, creating a new log object, and adding the identified starting key information text to the object;

[0171] S511, restore the removed nodes in the search tree and return to step S503.

[0172] Embodiment 3

[0173] Figure 6 This is a schematic diagram of the structure of a log information identification device provided by Embodiment 3 of the present invention. Figure 6 As shown, the device comprises:

[0174] A file acquisition module 61, used to acquire a log file to be processed and a pre-constructed key information search tree, wherein the key information search tree is constructed based on preset substring format information;

[0175] An information identification module 62, configured to identify key information of a substring of the log file to be processed based on the node substring structure of each node in the key information search tree to obtain key information;

[0176] The information saving module 63 is used to update the nodes in the key information search tree based on the key information and save the key information in the log object until the identification end condition is met, thereby obtaining all the log objects of the log file to be processed.

[0177] The technical solution of the embodiment of the present invention obtains the log file to be processed and the pre-constructed key information search tree, and the key information search tree is constructed based on the preset substring format information; based on the node substring structure of each node in the key information search tree, the substring of the log file to be processed is identified for key information to obtain key information; based on the key information, the nodes in the key information search tree are updated, and the key information is saved in the log object until the identification end condition is met, and all log objects of the log file to be processed are obtained. The key information search tree corresponding to the log file to be processed is constructed by the preset substring format information, and the key information in the log file to be processed is matched in the form of a substring through the key information search tree, and no longer matches characters one by one, thereby improving the matching speed, and no longer relying on a third-party lexical analysis program, thereby improving the convenience of the method.

[0178] Furthermore, the information identification module 62 is specifically used for:

[0179] Read the non-space characters currently traversed in the log file to be processed, and use the current log position of the characters in the log file to be processed as the starting identification position;

[0180] Determine the termination position according to the current root node of the key information search tree, the current number of child nodes corresponding to the current child node, and the node substring structure;

[0181] In the log file to be processed, based on the node substring structure, the substring of the interval corresponding to the start identification position and the end position is identified as key information, and the information category of the key information is determined;

[0182] In the key information search tree, nodes that meet the identified conditions are removed, and the traversal is continued until the traversal is completed.

[0183] Furthermore, the information storage module 63 is specifically used for:

[0184] When the information category of the key information is log content, the text of the key information is saved in the current log object, and the next unsaved key information is obtained;

[0185] When the information category of the key information is the starting key information, the current log object is saved as the previous log object, a new log object is created as the current log object and the starting key information is stored, and the nodes in the key information search tree are restored.

[0186] Optionally, the device further includes: a search tree construction module.

[0187] The search tree construction module comprises:

[0188] A template acquisition submodule, used to acquire the log file template of the log file to be processed;

[0189] A first determination submodule is used to classify the key information contained in the log file template according to the string representation format to obtain a key information substring containing a substring type;

[0190] A second determination submodule, used to determine a key information linked list according to the substring structure description and each of the key information substrings;

[0191] The search tree generation submodule is used to generate a key information search tree according to the key information linked list.

[0192] Furthermore, the second determining submodule is specifically used for:

[0193] Initialize a key information linked list consisting of the substring structure description;

[0194] For each of the key information substrings, if there is a starting constant substring in the substring type, the key information substring is divided into a plurality of first substrings according to characters, the first substring description information of the first substrings is changed, and each of the first substrings is sequentially added to the key information linked list according to a first order;

[0195] If there is an end constant substring in the substring type, dividing the key information substring into a plurality of second substrings according to characters, changing the second substring description information of the second substrings and sequentially adding each of the second substrings to the key information linked list according to a second sequence;

[0196] If there is a fixed-length non-constant substring in the substring type, the key information substring is divided into a plurality of substring structures whose length fields of the non-constant substrings have a value of 1, and each of the substring structures is added to the key information linked list;

[0197] If there is a variable-length non-constant substring in the substring type, the key information substring is represented by the substring structure and added to the key information linked list;

[0198] According to the last substring structure in the associated information linked list, the category value of the key information substring is determined, and the key information linked list filled with the key information substring is obtained.

[0199] Furthermore, the search tree generation submodule includes:

[0200] An initialization unit, used to determine the node structure of all nodes in the initial search tree according to the log file template, and convert the key information linked list into a tree node linked list under the node structure, and use the initial search tree as the current search tree; wherein the node structure of the root node is empty, and the node structure of the child node is the substring structure description;

[0201] A first determining unit is used to traverse the tree node linked list, use the tree node linked list that has not generated the search tree as the current linked list to be generated, and determine the linked list root node and linked list child nodes of the current linked list to be generated;

[0202] A second determination unit is used to determine whether the tree child node and the linked list root node meet an equality condition if the tree root node of the current search tree has a tree child node, and if the equality condition is met, determine whether the linked list root node has a linked list child node;

[0203] A third determining unit is used for adding the linked list root node to the tree child node list of the tree root node, updating the current search tree, and continuing to traverse the tree node linked list if the equality condition is not satisfied or the tree child node does not exist;

[0204] A fourth determination unit is used for, if a linked list child node exists, taking the tree child node as a tree root node, taking the linked list child node as the linked list root node, and continuing to perform the determination of whether a tree child node exists with the tree root node;

[0205] A fifth determining unit, configured to continue traversing the tree node linked list if no linked list child node exists;

[0206] The sixth determining unit is used to compress the current search tree when the traversal is completed to obtain a key information search tree.

[0207] Wherein, the sixth determining unit is specifically used for:

[0208] If the parent node in the current search tree has only one child node, and the substring structures of the parent node and the child node are equal, then merge the first substring structure of the child node into the second substring structure of the parent node to obtain a merged node, and delete the child node to obtain an updated search tree;

[0209] Taking nodes with the same depth in the update search tree as nodes to be sorted, and determining the third substring structure of each of the nodes to be sorted;

[0210] The nodes to be sorted are sorted according to the third substring structures to obtain a key information search tree.

[0211] The log information identification device provided in the embodiment of the present invention can execute the log information identification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0212] Embodiment 4

[0213] Figure 7 A schematic diagram of an electronic device 70 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0214] like Figure 7As shown, the electronic device 70 includes at least one processor 71, and a memory connected to the at least one processor 71, such as a read-only memory (ROM) 72, a random access memory (RAM) 73, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 71 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 72 or the computer program loaded from the storage unit 78 to the random access memory (RAM) 73. In the RAM 73, various programs and data required for the operation of the electronic device 70 can also be stored. The processor 71, the ROM 72, and the RAM 73 are connected to each other via a bus 74. An input / output (I / O) interface 75 is also connected to the bus 74.

[0215] A number of components in the electronic device 70 are connected to the I / O interface 75, including: an input unit 76, such as a keyboard, a mouse, etc.; an output unit 77, such as various types of displays, speakers, etc.; a storage unit 78, such as a disk, an optical disk, etc.; and a communication unit 79, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 79 allows the electronic device 70 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0216] The processor 71 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 71 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 71 executes the various methods and processes described above, such as the log information recognition method.

[0217] In some embodiments, the log information identification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 78. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 70 via the ROM 72 and / or the communication unit 79. When the computer program is loaded into the RAM 73 and executed by the processor 71, one or more steps of the log information identification method described above may be performed. Alternatively, in other embodiments, the processor 71 may be configured to perform the log information identification method in any other appropriate manner (e.g., by means of firmware).

[0218] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0219] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0220] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0221] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0222] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0223] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0224] In one embodiment, the embodiment of the present invention further includes a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the log information identification method of any embodiment of the present invention is implemented.

[0225] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0226] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0227] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A log information identification method, characterized in that: include: Obtaining a log file to be processed and a pre-constructed key information search tree, wherein the key information search tree is constructed based on preset substring format information; Based on the node substring structure of each node in the key information search tree, key information is identified on the substring of the log file to be processed to obtain key information; The nodes in the key information search tree are updated based on the key information, and the key information is stored in a log object until an identification end condition is met, thereby obtaining all the log objects of the log file to be processed.

2. The method according to claim 1, characterized in that The preset substring format information includes a string representation format and a substring structure description. Accordingly, the steps of constructing the key information search tree include: Obtain a log file template for the log file to be processed; Classifying the key information contained in the log file template according to the string representation format to obtain a key information substring containing a substring type; Determine a key information linked list according to the substring structure description and each of the key information substrings; A key information search tree is generated according to the key information linked list.

3. The method according to claim 2, characterized in that Determining the key information linked list according to the substring structure description and each of the key information substrings includes: Initialize a key information linked list consisting of the substring structure description; For each of the key information substrings, if there is a starting constant substring in the substring type, the key information substring is divided into a plurality of first substrings according to characters, the first substring description information of the first substrings is changed, and each of the first substrings is sequentially added to the key information linked list according to a first order; If there is an end constant substring in the substring type, dividing the key information substring into a plurality of second substrings according to characters, changing the second substring description information of the second substrings and sequentially adding each of the second substrings to the key information linked list according to a second sequence; If there is a fixed-length non-constant substring in the substring type, the key information substring is divided into a plurality of substring structures whose length fields of the non-constant substrings have a value of 1, and each of the substring structures is added to the key information linked list; If there is a variable-length non-constant substring in the substring type, the key information substring is represented by the substring structure and added to the key information linked list; According to the last substring structure in the associated information linked list, the category value of the key information substring is determined, and the key information linked list filled with the key information substring is obtained.

4. The method according to claim 2, characterized in that: The step of generating a key information search tree according to the key information linked list includes: According to the log file template, determine the node structure of all nodes in the initial search tree, convert the key information linked list into a tree node linked list under the node structure, and use the initial search tree as the current search tree; wherein the node structure of the root node is empty, and the node structure of the child node is the substring structure description; Traversing the tree node linked list, taking the tree node linked list for which the search tree has not been generated as the current linked list to be generated, and determining the linked list root node and the linked list child nodes of the current linked list to be generated; If the tree root node of the current search tree has a tree child node, determine whether the tree child node and the linked list root node meet the equality condition, and if the equality condition is met, determine whether the linked list root node has a linked list child node; If the equality condition is not satisfied or there is no tree child node, the linked list root node is added to the tree child node list of the tree root node, the current search tree is updated, and the tree node linked list is continued to be traversed; If there is a linked list child node, the tree child node is used as the tree root node, the linked list child node is used as the linked list root node, and the determination of whether there is a tree child node is continued with the tree root node; If there is no linked list child node, continue to traverse the tree node linked list; When the traversal is finished, the current search tree is compressed to obtain a key information search tree.

5. The method according to claim 4, characterized in that The compressing the current search tree to obtain a key information search tree includes: If the parent node in the current search tree has only one child node, and the substring structures of the parent node and the child node are equal, then merge the first substring structure of the child node into the second substring structure of the parent node to obtain a merged node, and delete the child node to obtain an updated search tree; Taking nodes with the same depth in the update search tree as nodes to be sorted, and determining the third substring structure of each of the nodes to be sorted; The nodes to be sorted are sorted according to the third substring structures to obtain a key information search tree.

6. The method according to claim 1, characterized in that The step of performing key information identification on the substring of the log file to be processed based on the node substring structure of each node in the key information search tree to obtain key information includes: Read the non-space characters currently traversed in the log file to be processed, and use the current log position of the characters in the log file to be processed as the starting identification position; Determine the termination position according to the current root node of the key information search tree, the current number of child nodes corresponding to the current child node, and the node substring structure; In the log file to be processed, based on the node substring structure, the substring of the interval corresponding to the start identification position and the end position is identified as key information, and the information category of the key information is determined; In the key information search tree, nodes that meet the identified conditions are removed, and the traversal is continued until the traversal is completed.

7. The method according to claim 1, characterized in that The updating of the nodes in the key information search tree based on the key information and storing the key information in a log object includes: When the information category of the key information is log content, the text of the key information is saved in the current log object, and the next unsaved key information is obtained; When the information category of the key information is the starting key information, the current log object is saved as the previous log object, a new log object is created as the current log object and the starting key information is stored, and the nodes in the key information search tree are restored.

8. A log information identification device, characterized in that: include: A file acquisition module, used to acquire a log file to be processed and a pre-built key information search tree, wherein the key information search tree is built based on preset substring format information; An information identification module, used to identify key information of the substring of the log file to be processed based on the node substring structure of each node in the key information search tree to obtain key information; The information saving module is used to update the nodes in the key information search tree based on the key information, and save the key information in the log object until the identification end condition is met, thereby obtaining all the log objects of the log file to be processed.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log information identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the log information identification method according to any one of claims 1 to 7 when executed.

Images