Log template acquisition method and apparatus, computer device, and storage medium

Abstract

The application provides a log template acquisition method and device, computer equipment and a storage medium, applied to the technical field of computers, and the method comprises the following steps: acquiring current log data, and performing preprocessing to obtain to-be-processed log data; performing word segmentation processing on each current log in the to-be-processed log data to obtain a current log word segmentation group corresponding to each current log; acquiring preset word frequency statistical information, and grouping each current log according to the preset word frequency statistical information and the current log word segmentation group corresponding to each current log to obtain a plurality of log groups, so that accurate grouping of the current log is realized, the problem that the current log classification is inaccurate due to different lengths of the current log of the log event in a variable-length variable log scene is effectively avoided, and then a target log template is acquired according to the preset word frequency statistical information, each log group and a preset log template database, so that the accuracy of log template extraction is improved.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a log template acquisition method, device, computer equipment and storage medium. Background Art

[0002] Computer systems, cloud servers, smartphones, laptops, and other systems often experience abnormal situations such as failures and security incidents during operation. Logs, as part of the operation of these systems, are one of the important ways for people to obtain these abnormal situations. Extracting log templates is an indispensable and important means for operation and maintenance work such as anomaly detection and fault diagnosis.

[0003] Currently, logs are mainly grouped and aggregated by the length of log segmentation words, and then log templates are extracted. However, log messages generated by different log events or the same log event may have the same log message length, resulting in the increase and rereading of log templates in variable-length variable log scenarios, resulting in low accuracy of extracted log templates.

[0004] The information included in this Background section is only for enhancement of understanding of the general background of the disclosure and should not be taken as an admission or any form of suggestion that this information constitutes prior art already known to a person skilled in the art. Summary of the Invention

[0005] The present application provides a log template acquisition method, apparatus, computer equipment and storage medium to improve the accuracy of log template extraction.

[0006] A log template acquisition method, the method comprising:

[0007] Get the current log data and pre-process it to get the log data to be processed;

[0008] Performing word segmentation processing on each current log in the log data to be processed to obtain a current log word group corresponding to each current log;

[0009] Obtaining preset word frequency statistical information, and grouping each current log according to the preset word frequency statistical information and the current log word group corresponding to each current log to obtain multiple log groups;

[0010] A target log template is obtained according to the preset word frequency statistical information, the log grouping and a preset log template database.

[0011] A log template acquisition device, the device comprising:

[0012] The first preprocessing module is used to obtain current log data and perform preprocessing to obtain log data to be processed;

[0013] A first word segmentation processing module is used to perform word segmentation processing on each current log in the log data to be processed to obtain a current log word group corresponding to each current log;

[0014] A log grouping module, configured to obtain preset word frequency statistics information, and group each current log according to the preset word frequency statistics information and the current log word group corresponding to each current log, to obtain a plurality of log groups;

[0015] The template determination module is used to obtain a target log template according to the preset word frequency statistical information, the log grouping and a preset log template database.

[0016] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the log template acquisition method is implemented.

[0017] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the log template acquisition method is implemented.

[0018] The above-mentioned log template acquisition method, device, computer equipment and storage medium obtain current log data and perform preprocessing to obtain log data to be processed; perform word segmentation processing on each current log in the log data to be processed to obtain the current log word group corresponding to each current log; obtain preset word frequency statistical information, and group each current log according to the preset word frequency statistical information and the current log word group corresponding to each current log to obtain multiple log groups; accurately group the current log by the preset word frequency statistical information and the current log word group of the current log, effectively avoiding the problem of inaccurate current log classification due to the different lengths of the current log of the log event in the variable-length variable log scenario, and then obtain the target log template according to the preset word frequency statistical information, each log group and the preset log template database, thereby improving the accuracy of log template extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 The following schematically illustrates an application environment diagram of a log template acquisition method provided in an embodiment of the present application.

[0021] Figure 2 A flow chart of a log template acquisition method provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of a process for obtaining preset word frequency statistics in a log template acquisition method provided in an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of the process of grouping current logs in a log template acquisition method provided in an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of a process for obtaining a target log template in a log template obtaining method provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of a process for obtaining a target log template in a log template obtaining method provided in another embodiment of the present application is shown;

[0026] Figure 7 A schematic diagram of the process of calculating the similarity value in the log template acquisition method provided in an embodiment of the present application is shown;

[0027] Figure 8 A schematic diagram of a process for obtaining a target log template in a log template obtaining method provided in another embodiment of the present application is shown;

[0028] Figure 9 A schematic diagram of the structure of a log template acquisition device provided in an embodiment of the present application is shown;

[0029] Figure 10 A schematic structural diagram of a computer device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0031] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0034] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0035] The log template acquisition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, Figure 1 As shown, the system framework 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. The server 105 may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0036] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0037] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Eperts Group Audio Layer III), MP4 players (Moving Picture Eperts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0038] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0039] It should be noted that the log template acquisition method provided in the embodiment of the present application can be executed by a server, and accordingly, the log template acquisition device is set in the server.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers is only illustrative. Depending on the implementation requirements, there can be any number of terminal devices, networks and servers. The terminal devices in the embodiments of the present application can specifically correspond to application systems in actual production.

[0041] In one embodiment, Figure 2 A flow chart showing a method for obtaining a log template in an embodiment of the present application is shown as follows: Figure 2 As shown, the log template acquisition method includes the following steps S100 to S400.

[0042] S100: Obtain current log data and perform pre-processing to obtain log data to be processed.

[0043] The current log data includes at least one current log, and the current log is a log generated by a log event at the current moment.

[0044] Specifically, the current log data is preprocessed based on preset processing rules to obtain log data to be processed; wherein, the preset processing rules can be obtained by analyzing historical experience data. For example, the preset processing rules may include deleting the log time, stop words, etc. in each current log in the current log data, replacing the mailbox, hash value, IP (Internet Protocol), WEB (World Wide Web), etc. The preset processing rules can be determined according to the actual application scenario and are not specifically limited here.

[0045] S200: Perform word segmentation processing on each current log in the log data to be processed to obtain a current log word group corresponding to each current log.

[0046] Among them, each current log in the log data to be processed is segmented based on preset symbols to obtain the current log segmentation group corresponding to each current log; exemplarily, the preset symbols may include comma (,), semicolon (;), period (.), space, etc. The preset symbols can be obtained by analyzing historical data and are not specifically limited here.

[0047] S300: Obtain preset word frequency statistical information, and group each current log according to the preset word frequency statistical information and a current log word segmentation group corresponding to each current log, to obtain a plurality of log groups.

[0048] The preset word frequency statistical information is obtained by analyzing historical log data, and includes word segmentation and a corresponding occurrence frequency in the historical log data. The preset word frequency statistical information can be pre-stored in the form of a data table.

[0049] S400: Obtain a target log template according to the preset word frequency statistical information, the log group, and a preset log template database.

[0050] The preset log template database can be an abstract syntax tree. The abstract syntax tree can be constructed based on the preset word frequency statistical information and the log template. For example, high-frequency word segmentation in the log template is determined according to the word frequency statistical information, and word segmentation labels of a root node to each leaf node in the abstract syntax tree are determined according to the order of the high-frequency word segmentation before and after the log template, so that the log template can be accurately obtained from the abstract syntax tree according to the word segmentation labels of the leaf node, which is beneficial to improving the accuracy of log template extraction. It should be particularly noted that the depth of the abstract syntax tree is determined according to an actual application scenario, which is not limited here.

[0051] The high-frequency word segmentation in the log group can be determined according to the preset word frequency statistical information, and the target log template associated with the leaf node can be obtained by matching the occurrence order of the high-frequency word segmentation in the log group and the word segmentation labels of the nodes in the abstract syntax tree until the leaf node is reached.

[0052] In this embodiment, the current log is accurately grouped by the preset word frequency statistical information and the current log word segmentation group of the current log, effectively avoiding the problem of inaccurate classification of the current log due to different lengths of the current log of the log event in the variable-length variable log scenario. Then, the target log template is obtained according to the preset word frequency statistical information, the log group, and the preset log template database, thereby improving the accuracy of log template extraction.

[0053] In some optional implementations of this embodiment, Figure 3 A flowchart of obtaining the preset word frequency statistical information in the log template obtaining method provided by an embodiment of the present application is shown. As shown in Figure 3 Before step S300, before obtaining the preset word frequency statistical information, the method further includes steps S310 to S312.

[0054] S310: Obtain historical log data and pre-process the historical log data to obtain processed historical log data.

[0055] The current log data includes historical logs generated within a preset time period before the current moment. The preset time period can be one month, one year, or one quarter, which is determined according to the actual application scenario and is not specifically limited here.

[0056] Specifically, the historical log data is preprocessed based on preset processing rules to obtain the historical log data to be processed; wherein, the preset processing rules can be obtained by analyzing historical experience data. For example, the preset processing rules may include deleting the log time, stop words, etc. in each historical log in the historical log data, replacing the mailbox, hash value, IP (Internet Protocol), WEB (World Wide Web), etc. The preset processing rules can be determined according to the actual application scenario and are not specifically limited here.

[0057] S311: Perform word segmentation processing on the historical log data to be processed to obtain historical log word segmentation.

[0058] Among them, each historical log in the historical log data to be processed is segmented based on preset symbols to obtain the historical log segmentation corresponding to each historical log; exemplarily, the preset symbols may include comma (,), semicolon (;), period (.), space, etc. The preset symbols can be obtained based on the analysis of historical data, and are not specifically limited here.

[0059] S312: Perform word frequency statistics on the historical log word segments to obtain preset word frequency statistical information.

[0060] Specifically, the frequency of occurrence of each historical word in the historical log word segmentation is counted to obtain preset word frequency statistical information.

[0061] In this embodiment, by performing word frequency statistics on each historical word after word segmentation on the historical log data, preset word frequency statistical information is obtained, which is conducive to subsequent accurate grouping of the current log, thereby improving the accuracy of log template extraction.

[0062] In some optional implementations of this embodiment, Figure 4 The following is a flow chart showing the process of grouping the current log in the log template acquisition method provided by an embodiment of the present application. Figure 4 As shown, in step S300, grouping each current log according to the preset word frequency statistical information and the current log word group corresponding to each current log includes the following steps S320 to S322.

[0063] S320: Obtain a first preset parameter, and determine the word segmentation information to be compared in the current log word segmentation group corresponding to each current log according to the first preset parameter.

[0064] The first preset parameter is a constant, which is determined according to the actual application scenario and is not specifically limited here;

[0065] For example, assuming that the first preset parameter is 4, the first four segmented words in the current log segmented word group corresponding to each current log are found as the segmented words to be compared.

[0066] S321: Determine the grouping word segmentation information corresponding to each current log according to the preset word frequency statistics information and the word segmentation information to be compared corresponding to each current log.

[0067] Specifically, the historical segmentation words in the preset word frequency statistics information are matched with the segmentation words to be compared in the segmentation word information to be compared to obtain a matching result, and the matching result is used to characterize the historical segmentation words matched by the segmentation words to be compared; the occurrence frequency corresponding to the matched historical segmentation words in the preset word frequency statistics information is used as the number of occurrences of the segmentation words to be compared, and the grouped segmentation information corresponding to each current log is determined according to the preset second parameter and the number of occurrences of each segmentation word to be compared. For example, assuming that the preset second parameter is 3, the number of occurrences of each segmentation word to be compared is sorted from large to small, and the first 3 segmentation words to be compared are used as the grouped segmentation information.

[0068] S322: Group each current log according to the grouping and word segmentation information corresponding to each current log to obtain multiple log groups.

[0069] Specifically, current logs with the same grouping word segmentation information are determined as a group to obtain multiple log groups.

[0070] In this embodiment, the grouping and word segmentation information in the current log is determined by using preset word frequency statistics information, so that the current log is accurately grouped according to the grouping and word segmentation information, thereby improving the accuracy of log template extraction.

[0071] In some optional implementations of this embodiment, Figure 5 FIG. 1 shows a flow chart of obtaining a target log template in a log template obtaining method provided in an embodiment of the present application. Figure 5 As shown, in step S400, the preset log template database includes multiple index information and log templates associated with each index information; according to the preset word frequency statistics, log grouping and the preset log template database, obtaining the target log template includes the following steps S410 to S412:

[0072] S410: Determine target index information in a preset log template database according to the current log word group and index information corresponding to each current log included in the log group.

[0073] Among them, the target index information is the index information matched by the current log, the preset log template database can be an abstract syntax tree, and the index information is composed of the word segmentation labels on the nodes between the root node and the leaf node of the abstract syntax tree. The current log word group corresponding to each current log contained in the log group is matched with the index information to determine the target index information. For the explanation of the abstract syntax tree, please refer to the previous embodiment and will not be repeated here.

[0074] S411: According to the target index information, a log template associated with the target index information is obtained from a preset log template database as a log template to be compared.

[0075] S412: Obtain a target log template according to the current log word group corresponding to each current log included in the log group, preset word frequency statistics information, and the log template to be compared.

[0076] Among them, the frequency of occurrence of the current log word corresponding to each current log contained in the log group can be determined based on the preset word frequency statistical information, and the current log words whose occurrence frequency is greater than the preset frequency threshold are combined into a word sequence to be compared. Based on the similarity algorithm, the word sequence to be compared is matched with the log template to be compared; if the match is successful, the log template to be compared is updated according to the current log word group corresponding to the current log to obtain the target log template; if the match is unsuccessful, a new log template is created according to the current log word group corresponding to the current log as the target log template.

[0077] In this embodiment, the target index information is determined based on the current log word group and index information corresponding to each current log contained in the log grouping. By determining the target index information based on the log grouping, the accuracy and efficiency of obtaining the log template to be compared are improved. Then, the target log template is obtained based on the current log word group corresponding to each current log contained in the log grouping, the preset word frequency statistics information and the log template to be compared, thereby improving the accuracy of log template extraction.

[0078] In some optional implementations of this embodiment, Figure 6 FIG. 1 shows a flow chart of obtaining a target log template in a log template obtaining method provided in another embodiment of the present application. Figure 6 As shown, in step S412, according to the current log word group corresponding to each current log included in the log group, the preset word frequency statistics information and the log template to be compared, obtaining the target log template includes the following steps S4120 to S4121:

[0079] S4120: Determine a similarity value between the log group and the log template to be compared based on the current log word group corresponding to each current log included in the log group, preset word frequency statistics information, and the log template to be compared.

[0080] Among them, the high-frequency segmented words in the current log segmented word group corresponding to each current log contained in the log group can be determined according to the preset word frequency statistical information, as the high-frequency segmented word sequence, and the similarity between the high-frequency segmented word sequence and the log template to be compared is calculated based on the similarity algorithm to obtain the similarity value.

[0081] S4121: Obtain a target log template according to the similarity value, the log template to be compared, and the current log word group corresponding to each current log included in the log group.

[0082] Specifically, based on the similarity value, determine whether to use the current log word group corresponding to each current log contained in the log group to update the log template to be compared as the target log template. If not, use the current log word group corresponding to each current log contained in the log group to create a new log template as the target log template.

[0083] In this embodiment, the log template to be compared is processed by determining the similarity value between the log group and the log template to be compared, thereby obtaining the target log template. This can avoid the problem of low accuracy in log template extraction caused by subtle differences in logs when the system is constantly updated, thereby improving the accuracy of log template extraction.

[0084] In some optional implementations of this embodiment, Figure 7 The following is a flow chart showing the process of calculating the similarity value in the log template acquisition method provided in one embodiment of the present application. Figure 7 As shown, in step S4120, determining the similarity value between the log group and the log template to be compared based on the current log word group corresponding to each current log contained in the log group, the preset word frequency statistics information and the log template to be compared includes the following steps S41200 to S41203:

[0085] S41200: Input the current log word group corresponding to each current log contained in the log group into the pre-trained neural network model for training, and obtain the position vector and word meaning vector corresponding to each word in each current log as the first position vector and the first word meaning vector.

[0086] Among them, the pre-trained neural network model can be a BERT model.

[0087] S41201: Input the log template to be compared into a pre-trained neural network model for training, and obtain the position vector and word sense vector of each word in the log template to be compared as the second position vector and the second word sense vector.

[0088] S41202: Determine the number of high-frequency words and low-frequency words included in the log group based on word frequency statistics.

[0089] The number of high-frequency words is the number of high-frequency words contained in the log group, and the number of low-frequency words is the number of low-frequency words contained in the log group.

[0090] It should be noted here that the high frequency and low frequency in the high-frequency participles, low-frequency participles, number of high-frequency participles, and number of low-frequency participles mentioned in this application are determined according to the actual application scenario and are not specifically limited here. For example, in one scenario, when the number of occurrences of a participle is greater than 9, it is a high frequency, and when the number of occurrences of a participle is less than and equal to 9, it is a low frequency; and in another scenario, when the number of occurrences of a participle is greater than 30, it is a high frequency, and when the number of occurrences of a participle is less than and equal to 30, it is a low frequency.

[0091] S41203: Calculate the first position vector, the first word meaning vector, the second position vector, the second word meaning vector, the number of high-frequency word segments, and the number of low-frequency word segments to obtain a similarity value.

[0092] Specifically, the similarity value is calculated according to the following formula (1):

[0093]

[0094] Where α and β are weights, and α+β=1, M is the number of high-frequency words contained in the log group, P is the number of low-frequency words contained in the log group, F(A i ) is the word A in the log group i The first position vector, V(A i ) is the word A in the log group i The first word sense vector, F(B i ) is the segmentation A in the template to be compared i The first position vector, V(B i ) is the segmentation A in the price comparison template i The first sense vector of is the set of low-frequency words in the log grouping, is the set of low-frequency words in the template to be compared.

[0095] In an embodiment, the current log word group corresponding to each current log contained in the log group and the log template to be compared are input into a preset neural network model for training to obtain a first word meaning vector, a first position vector, a second word meaning vector and a second position vector, and the similarity is calculated based on the above vectors and the number of low-frequency words and the number of high-frequency words, thereby weakening the influence of the variable position and strengthening the importance of word segmentation in the log, effectively ensuring that the log template has the most reasonable degree of abstraction, and improving the accuracy of log template extraction. In addition, it can also meet the needs of different application scenarios and effectively improve generalization.

[0096] In some optional implementations of this embodiment, Figure 8 FIG. 1 shows a flow chart of obtaining a target log template in a log template obtaining method provided in another embodiment of the present application. Figure 8 As shown, in step S4121, obtaining the target log template according to the similarity value, the log template to be compared, and the log word group corresponding to each current log contained in the log group includes the following steps S41210 to S41211:

[0097] S41210: If the similarity value is greater than a preset threshold, the log template to be compared is updated according to the log word group corresponding to each current log included in the log group to obtain a target log template.

[0098] Specifically, the high-frequency words in the log word group corresponding to each current log contained in the log group are matched with the words in the log template to be compared, unmatched words are determined, and the unmatched words in the log template to be compared are replaced based on a preset wildcard, which can be "*".

[0099] S41211: If the similarity value is less than and equal to a preset threshold, a new log template is created according to the log word group corresponding to each current log included in the log group as a target log template.

[0100] In this embodiment, judgment is made through similarity values, and the log template to be compared is processed according to the log word groups corresponding to each current log contained in the log grouping, so as to obtain the target log template. This can avoid the problem of low accuracy in log template extraction caused by subtle differences in logs when the system is constantly updated, and improve the accuracy of log template extraction.

[0101] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0102] In one embodiment, a log template acquisition device is provided, which corresponds to the log template acquisition method in the above embodiment. For details not disclosed in the embodiment of the device of this application, please refer to the embodiment of the log template acquisition method in the above embodiment of this application. Figure 9 As shown, the log template acquisition device includes a first pre-processing module 30, a first word segmentation processing module 31, a log grouping module 32 and a template determination module 33. The functional modules are described in detail as follows:

[0103] The first pre-processing module 30 is used to obtain current log data and perform pre-processing to obtain log data to be processed.

[0104] The first word segmentation processing module 31 is configured to perform word segmentation processing on each current log in the log data to be processed, and obtain a current log word segmentation group corresponding to each current log.

[0105] The log grouping module 32 is configured to obtain preset word frequency statistics information, and group each current log according to the preset word frequency statistics information and the current log word group corresponding to each current log to obtain multiple log groups.

[0106] The template determination module 33 is used to obtain a target log template according to preset word frequency statistics, log grouping and a preset log template database.

[0107] Optionally, the log template obtaining device further includes:

[0108] The second preprocessing module is used to obtain historical log data and perform preprocessing to obtain historical log data to be processed.

[0109] The second word segmentation processing module is used to perform word segmentation processing on the historical log data to be processed to obtain historical log word segments.

[0110] The word frequency statistics module is used to perform word frequency statistics on historical log word segmentation to obtain preset word frequency statistics information.

[0111] Optionally, the log grouping module 32 includes:

[0112] The parameter acquisition submodule is used to obtain a first preset parameter and determine the word segmentation information to be compared in the current log word segmentation group corresponding to each current log according to the first preset parameter.

[0113] The information determination submodule is used to determine the grouping and word segmentation information corresponding to each current log according to the preset word frequency statistical information and the word segmentation information to be compared corresponding to each current log.

[0114] The log grouping submodule is used to group each current log according to the grouping and word segmentation information corresponding to each current log to obtain multiple log groups.

[0115] Optionally, the preset log template database includes multiple index information and log templates associated with each index information, and the template determination module 33 includes:

[0116] The target index information determination submodule is used to determine the target index information in the preset log template database according to the current log word group and index information corresponding to each current log included in the log group.

[0117] The log template to be compared determining submodule is used to obtain the log template associated with the target index information from a preset log template database according to the target index information as the log template to be compared.

[0118] The log template determination submodule is used to obtain a target log template according to the current log word group corresponding to each current log included in the log group, preset word frequency statistics and the log template to be compared.

[0119] Optionally, the log template determination submodule includes:

[0120] The similarity value determining unit is used to determine the similarity value between the log group and the log template to be compared according to the current log word group corresponding to each current log contained in the log group, preset word frequency statistics information and the log template to be compared.

[0121] The log template determining unit is used to obtain a target log template according to the similarity value, the log template to be compared, and the current log word group corresponding to each current log included in the log group.

[0122] Optionally, the similarity value determining unit includes:

[0123] The first vector determination subunit is used to input the current log word group corresponding to each current log contained in the log group into the pre-trained neural network model for training, and obtain the position vector and word sense vector corresponding to each word in each current log as the first position vector and the first word sense vector.

[0124] The second vector determination subunit is used to input the log template to be compared into the pre-trained neural network model for training, and obtain the position vector and word sense vector of each word in the log template to be compared as the second position vector and the second word sense vector.

[0125] The quantity determination subunit is used to determine the number of high-frequency words and low-frequency words contained in the log group according to word frequency statistics information.

[0126] The similarity value determination subunit is used to calculate the first position vector, the first word meaning vector, the second position vector, the second word meaning vector, the number of high-frequency word segments and the number of low-frequency word segments to obtain a similarity value.

[0127] Optionally, the log template determining unit includes:

[0128] The first log template determining subunit is configured to update the log template to be compared according to the log word group corresponding to each current log included in the log group to obtain a target log template if the similarity value is greater than a preset threshold.

[0129] The second log template determining subunit is configured to create a new log template as a target log template according to the log word group corresponding to each current log included in the log group if the similarity value is less than and equal to a preset threshold.

[0130] The meaning of "first" and "second" in the above modules / units is only to distinguish different modules / units, and is not used to limit which module / unit has a higher priority or other limiting meanings. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules in this application is only a logical division, and other division methods may be used in actual applications.

[0131] For the specific definition of the log template acquisition device, please refer to the definition of the log template acquisition method above, which will not be repeated here. The various modules in the above-mentioned log template acquisition device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0132] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the log template acquisition method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a log template acquisition method is implemented.

[0133] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the log template acquisition method in the above embodiment are implemented, such as Figure 2 Alternatively, when the processor executes the computer program, the functions of each module / unit of the log template acquisition device in the above embodiment are realized, for example Figure 9 The functions of modules 30 to 33 are shown in FIG. 3 and will not be described in detail here to avoid repetition.

[0134] The processor can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device and connects the various parts of the entire computer device using various interfaces and lines.

[0135] The memory can be used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data generated based on the use of the mobile phone (such as audio data, video data, etc.).

[0136] The memory may be integrated into the processor or provided separately from the processor.

[0137] In one embodiment of the present application, a computer-readable storage medium is further provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by the processor of the computer, the computer executes the above-mentioned log template acquisition method. In particular, according to an embodiment of the present application, the process described in each method flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit, the various functions defined in the system of the present application are executed.

[0138] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0139] In one embodiment, a computer-readable storage medium is provided on which a computer program is stored. When the computer program is executed by a processor, the steps of the log template acquisition method in the above embodiment are implemented, for example Figure 2 Alternatively, when the computer program is executed by the processor, the functions of each module / unit of the log template acquisition device in the above embodiment are realized, such as Figure 9 The functions of modules 30 to 33 are shown in FIG. 3 and will not be described in detail here to avoid repetition.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0141] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art that are not disclosed herein.

[0142] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A log template acquisition method, characterized in that: The method comprises: Get the current log data and pre-process it to get the log data to be processed; Performing word segmentation processing on each current log in the log data to be processed to obtain a current log word group corresponding to each current log; Obtaining preset word frequency statistical information, and grouping each current log according to the preset word frequency statistical information and the current log word group corresponding to each current log to obtain multiple log groups; Obtaining a target log template according to the preset word frequency statistical information, the log grouping, and a preset log template database; Among them, obtaining preset word frequency statistical information, and grouping each current log according to the preset word frequency statistical information and the current log word group corresponding to each current log to obtain multiple log groups, including: obtaining a first preset parameter, and determining the to-be-compared word segmentation information in the current log word group corresponding to each current log according to the first preset parameter; determining the grouped word segmentation information corresponding to each current log according to the preset word frequency statistical information and the to-be-compared word segmentation information corresponding to each current log; grouping each current log according to the grouped word segmentation information corresponding to each current log to obtain multiple log groups.

2. The log template acquisition method according to claim 1, characterized in that: Before obtaining the preset word frequency statistical information, the method further includes: Obtain historical log data and perform preprocessing to obtain the historical log data to be processed; Performing word segmentation processing on the historical log data to be processed to obtain historical log word segmentation; Perform word frequency statistics on the historical log word segmentation to obtain the preset word frequency statistical information.

3. The log template acquisition method according to claim 1, characterized in that: The preset log template database includes a plurality of index information and a log template associated with each index information; The acquiring of a target log template according to the preset word frequency statistical information, the log grouping, and a preset log template database includes: Determining target index information in the preset log template database according to the current log word group and the index information corresponding to each current log included in the log group; According to the target index information, obtaining a log template associated with the target index information from the preset log template database as the log template to be compared; The target log template is acquired according to the current log word group corresponding to each current log included in the log group, the preset word frequency statistics information and the log template to be compared.

4. The log template acquisition method according to claim 3, characterized in that: The acquiring the target log template according to the current log word group corresponding to each current log included in the log group, the preset word frequency statistics information, and the log template to be compared includes: Determining a similarity value between the log group and the log template to be compared based on the current log word group corresponding to each current log contained in the log group, the preset word frequency statistical information, and the log template to be compared; The target log template is obtained according to the similarity value, the log template to be compared, and the current log word group corresponding to each current log included in the log group.

5. The log template acquisition method according to claim 4, characterized in that: The determining of the similarity value between the log group and the log template to be compared based on the current log word group corresponding to each current log contained in the log group, the preset word frequency statistics information, and the log template to be compared includes: Inputting the current log word group corresponding to each current log contained in the log group into a pre-trained neural network model for training, and obtaining a position vector and a word sense vector corresponding to each word in each current log as a first position vector and a first word sense vector; Inputting the log template to be compared into the pre-trained neural network model for training, obtaining a position vector and a word sense vector of each word in the log template to be compared as a second position vector and a second word sense vector; Determine the number of high-frequency words and the number of low-frequency words included in the log group according to the word frequency statistical information; The first position vector, the first word meaning vector, the second position vector, the second word meaning vector, the number of high-frequency word segments, and the number of low-frequency word segments are calculated to obtain the similarity value.

6. The log template acquisition method according to claim 4, characterized in that: The acquiring the target log template according to the similarity value, the log template to be compared, and the log word group corresponding to each current log included in the log grouping includes: If the similarity value is greater than a preset threshold, the log template to be compared is updated according to the log word group corresponding to each current log contained in the log group to obtain the target log template; If the similarity value is less than and equal to the preset threshold, a new log template is created according to the log word group corresponding to each current log included in the log group as the target log template.

7. A log template acquisition device, characterized in that: The device comprises: The first preprocessing module is used to obtain current log data and perform preprocessing to obtain log data to be processed; A first word segmentation processing module is used to perform word segmentation processing on each current log in the log data to be processed to obtain a current log word group corresponding to each current log; A log grouping module, configured to obtain preset word frequency statistics information, and group each current log according to the preset word frequency statistics information and the current log word group corresponding to each current log, to obtain a plurality of log groups; A template determination module, configured to obtain a target log template based on the preset word frequency statistical information, the log grouping, and a preset log template database; Among them, the log grouping module is also used to obtain a first preset parameter, and determine the to-be-compared word segmentation information in the current log word segmentation group corresponding to each current log according to the first preset parameter; determine the grouping word segmentation information corresponding to each current log according to the preset word frequency statistics information and the to-be-compared word segmentation information corresponding to each current log; group each current log according to the grouping word segmentation information corresponding to each current log to obtain multiple log groups.

8. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the log template acquisition method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the log template acquisition method according to any one of claims 1 to 6 is implemented.

Images