Log analysis method and apparatus, electronic device, and medium

Abstract

The application discloses a log analysis method and device, electronic equipment and medium, and belongs to the technical field of communication. The method comprises the following steps: performing structural processing on log data to be analyzed to obtain log events corresponding to the log data; splicing a plurality of log events to obtain an event text sequence; inputting the event text sequence into a target prediction model corresponding to a target prediction task for processing to obtain a target processing result of the log data on the target prediction task; wherein the target prediction model comprises an encoding model and a decoding model, and the encoding model is obtained by training a preset pre-training language model by using sample log data corresponding to the log data and a log text vocabulary. The method can accurately and efficiently analyze the log data and improve the accuracy of the obtained target processing result.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and more specifically to a log analysis method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] During the daily operation of communication equipment, it typically outputs large-scale log data, which records the operating information of the communication equipment.

[0003] To ensure the normal operation of communication equipment, the equipment status is typically determined manually by analyzing equipment log data. For example, maintenance personnel actively query and analyze log data to ascertain the equipment's operational status. However, this log analysis method often suffers from inaccuracies and inefficiencies. To address this, pre-trained language models, such as Transformer and BERT, are used to analyze communication equipment log data. However, communication equipment log data often exhibits strong domain-specific characteristics, and most existing pre-trained language models are trained using existing corpora. Therefore, analyzing communication equipment log data using pre-trained language models trained in this way often results in inaccuracies. Summary of the Invention

[0004] Therefore, the present invention provides a log analysis method, apparatus, electronic device and medium to solve the problem of inaccuracy in the analysis of log data of communication devices in the prior art.

[0005] To achieve the above objectives, a first aspect of the present invention provides a log analysis method, the log analysis method comprising:

[0006] The log data to be analyzed is structured to obtain the log events corresponding to the log data;

[0007] Concatenate multiple log events to obtain an event text sequence;

[0008] The event text sequence is input into the target prediction model corresponding to the target prediction task for processing, and the target processing result of the log data on the target prediction task is obtained.

[0009] The target prediction model includes an encoding model and a decoding model. The encoding model is trained on a pre-trained language model using sample log data and a log text vocabulary corresponding to the log data.

[0010] Optionally, the target prediction task includes an event prediction task;

[0011] The step of inputting the event text sequence into the target prediction model corresponding to the target prediction task for processing, and obtaining the target processing result of the log data on the target prediction task, includes:

[0012] The event text sequence is input into the encoding model for event encoding and event prediction processing to obtain a predicted event feature sequence; wherein, the predicted event feature sequence includes feature data of at least one predicted log event associated with the log event in the event text sequence;

[0013] The predicted event feature sequence is input into the decoding model of the target prediction model for log event generation processing to obtain the at least one predicted log event;

[0014] The target processing result is obtained based on the at least one predicted log event.

[0015] Optionally, the encoding model is trained through the following steps:

[0016] The sample log data is structured and concatenated to obtain a sample event text sequence;

[0017] The encoding model is trained based on the sample event text sequence and the log text vocabulary to obtain the encoding model.

[0018] Optionally, the encoding model includes a text encoding sub-model and an event encoding sub-model;

[0019] The step of training the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model includes:

[0020] The sample event text sequence is segmented based on the log text vocabulary.

[0021] At least one word of any log event in the sample event text sequence is masked to obtain a first masked text sequence;

[0022] The sample event text sequence and the first occluded text sequence are respectively input into the text encoding sub-model to obtain the first text feature and the first occluded text feature;

[0023] Based on the first text features and the first masked text features, the first loss of the encoding model is obtained;

[0024] Masking the sub-features in the first text feature corresponding to any log event to obtain the second text feature; and reordering the sub-features in the first text feature corresponding to each log event to obtain the third text feature.

[0025] The second text feature and the third text feature are respectively input into the event encoding sub-model to obtain the first event feature and the second event feature. The first event feature is the feature obtained by the event encoding sub-model after predicting the occluded sub-features in the second text feature, and the second event feature is the feature obtained by the event encoding sub-model after reordering the sub-features in the third text feature.

[0026] Based on the first text features, the first event features, and the second event features, a second loss value is obtained;

[0027] Based on the first loss value and the second loss value, the parameters of the encoding model are adjusted to obtain the encoding model that satisfies the preset convergence condition.

[0028] Optionally, obtaining the second loss value based on the first text feature, the first event feature, and the second event feature includes:

[0029] Obtain a third loss between the first text feature and the first event feature; and obtain a fourth loss between the first text feature and the second event feature;

[0030] The second loss is obtained based on the third loss and the fourth loss.

[0031] Optionally, obtaining the first loss of the encoding model based on the first text features and the first masked text features includes:

[0032] The first text feature and the first occluded text feature are respectively input into the first perception network to obtain the first sample score and the first occlusion score of the first text feature and the first occlusion text feature on the log text vocabulary;

[0033] The first loss is obtained based on the first sample score and the first occlusion score.

[0034] Optionally, obtaining the third loss between the first text feature and the first event feature includes:

[0035] From the first text features, obtain the standard event features corresponding to the target log event, wherein the target log event is the log event corresponding to the occluded sub-feature in the second text features;

[0036] Obtain the predicted event features corresponding to the target log event from the first event features;

[0037] The third loss is obtained based on the standard event characteristics and the predicted event characteristics.

[0038] Optionally, obtaining the fourth loss between the first text features and the second event features includes:

[0039] The first text feature and the second event feature are respectively input into the second perceptual network to obtain a first confidence sequence and a second confidence sequence, wherein the first confidence sequence is used to represent the confidence distribution of the log event sequence corresponding to the first text feature, and the second confidence sequence is used to represent the confidence distribution of the log event sequence represented by the second event feature.

[0040] The fourth loss is obtained based on the first confidence sequence and the second confidence sequence.

[0041] To achieve the above objectives, a second aspect of the present invention also provides a log analysis apparatus, the apparatus comprising:

[0042] The structured processing module is used to perform structured processing on the log data to be analyzed, and obtain the log events corresponding to the log data.

[0043] The event concatenation module is used to concatenate multiple log events to obtain an event text sequence;

[0044] The analysis module is used to input the event text sequence into the target prediction model corresponding to the target prediction task for processing, and to obtain the target processing result of the log data on the target prediction task;

[0045] The target prediction model includes an encoding model and a decoding model. The encoding model is trained on a pre-trained language model using sample log data and a log text vocabulary corresponding to the log data.

[0046] To achieve the above objectives, a third aspect of the present invention also provides an electronic device, comprising:

[0047] One or more processors;

[0048] A memory having stored one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the log analysis method described in the first aspect of the present invention;

[0049] One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

[0050] To achieve the above objectives, a fourth aspect of the present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the log analysis method described in the first aspect of the present invention.

[0051] The present invention has the following advantages: It obtains log events corresponding to the log data by performing structured processing on the log data to be analyzed. Then, considering the temporal dependencies between log events in communication devices, it concatenates multiple log events in the log data to obtain an event text sequence. This event text sequence is then input into the target prediction model corresponding to the target prediction task for processing. Since the encoding model included in the target prediction model is trained on a pre-trained language model using sample log data and a log text vocabulary corresponding to the log data, the target prediction model can accurately extract structural features, temporal features, and other domain features of the log data to analyze the log data, thereby accurately obtaining the target processing result of the log data on the target prediction task. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0053] Figure 1 A flowchart illustrating a log analysis method provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of the coding model provided in an embodiment of the present invention;

[0055] Figure 3a This is a schematic diagram of the process for training an encoding model provided in an embodiment of the present invention;

[0056] Figure 3b This is a schematic diagram of a framework for training an encoding model provided in an embodiment of the present invention;

[0057] Figure 4 A block diagram of a log analysis device provided in an embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0060] As used in this invention, the term "and / or" includes any and all combinations of one or more of the associated enumerated entries.

[0061] The terminology used in this invention is for describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0062] When the terms “comprising” and / or “made of” are used in this invention, the presence of the said feature, integral, step, operation, element and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or groups thereof is not excluded.

[0063] Unless otherwise specified, all terms used in this invention (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined by the invention.

[0064] To improve the accuracy and efficiency of log analysis, a log analysis model can be trained using the following steps: 1. Train a pre-trained language model using massive amounts of natural language text as corpus, such as Transformer or BERT; 2. Fine-tune the pre-trained language model trained in the previous step using historical log data from communication devices as sample data to obtain the log analysis model; After obtaining the log analysis model, input the log data from the communication devices into the log analysis model for log analysis processing.

[0065] However, considering that current word frequency-based segmentation methods, such as WordPiece, are generally used for natural language text, the fixed tokens in communication device log data often have strong domain knowledge constraints. Taking the token "CMDCONFIRM_CHANGEPWDCHOOSEINFO" in a log data of a communication device, "GSG-8User=gzunicom IP=198.xx.xx.xVpnName=SHELL CMDCONFIRM_CHANGEPWDCHOOSEINFO", as an example, when the log analysis model obtained based on the above steps performs log analysis on this log data, if the word frequency-based segmentation method is used in the model, it will split the complete token "CMDCONFIRM_CHANGEPWDCHOOSEINFO" into multiple tokens such as "CMD", "CONFIRM", "CHANGE", "PWD", "CHOOSE", and "INFO", which have no actual business meaning. Therefore, the model cannot correctly understand the actual business meaning represented by the complete token, and may thus draw incorrect log analysis results.

[0066] Furthermore, compared to natural language text, log data from communication devices often exhibits strong structure. For example, fields at different locations within each log entry often display similar structural patterns. Therefore, log data structures have multiple levels, such as event level, device level, and module level. Currently, when training log analysis models, historical log data from communication devices is typically obtained and directly used as ordinary natural language text to construct the training set. The pre-trained language model is then fine-tuned based on this training set. This means that the model may not correctly learn the structural features of the log data during fine-tuning, potentially leading to the model completely ignoring these structural features during log analysis and thus providing incorrect analysis results.

[0067] Furthermore, log data from communication devices often exhibits temporal sequence characteristics. For example, the occurrence of a particular log entry may be dependent on a log entry from a previous time. However, during the current model fine-tuning phase, this characteristic of log data is often not specifically trained, causing the resulting log analysis model to ignore the temporal dependencies between log data when performing log analysis, thus failing to provide accurate analysis results.

[0068] In view of this, in this embodiment of the invention, for the log data to be analyzed, the log data is first subjected to structured processing to obtain log events that can represent the structural characteristics of the log data; then, the multiple log events are concatenated to obtain an event text sequence that can characterize the temporal dependencies between the log events, and the event text sequence is input into the target prediction model corresponding to the target prediction task for processing. Since the encoding model included in the target prediction model is trained on a preset pre-trained language model using sample log data and a log text vocabulary corresponding to the log data, the target prediction model can analyze the log data while correctly extracting structural features and temporal features, and thus accurately obtain the target processing result of the log data on the target prediction task.

[0069] Please refer to Figure 1 This is a flowchart illustrating a log analysis method provided by an embodiment of the present invention. The method can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0070] like Figure 1 The log analysis method provided in this embodiment of the invention may include the following steps S101-S103, which will be described in detail below.

[0071] Step S101: Perform structured processing on the log data to be analyzed to obtain the log events corresponding to the log data.

[0072] In this embodiment of the invention, the log data can be data generated by the communication device during operation, recording various log events. This log data often has specific structural characteristics and time-series dependencies. It should be noted that the communication device in this embodiment of the invention can be, for example, a router, switch, modem, wireless bridge, or other wired or wireless communication devices, or other devices, without special limitation here.

[0073] For example, the log data in this embodiment of the invention can be composed of the fields shown in Table 1 below. Of course, this is only an example, and in actual implementation, the field structure of the log data can be modified as needed, without any special limitation here.

[0074] Table 1

[0075]

[0076] In some embodiments, the structuring of the log data to be analyzed described in step S101 may include: step S1011, performing data cleaning and preprocessing on the log data; and step S1012, extracting key-value pairs from the key information in the log data after data cleaning and preprocessing to obtain structured log data.

[0077] The data cleaning process can be used to filter out log data with abnormal values, such as log data with obviously abnormal timestamps or log data with garbled characters. The preprocessing can be used to denoise the log data, such as removing stop words, noisy characters, etc., which may be present in the log data. Noisy characters can be HTML tags, special characters, etc., which interfere with log analysis.

[0078] Key information in log data refers to the fields that constitute the log data and their corresponding values. In this embodiment of the invention, after extracting key-value pairs from the key information in the log data after data cleaning and preprocessing, the resulting structured log data may include the following:

[0079] {'time':1551122367,

[0080] 'host_name':'CSG-8',

[0081] 'des':'Failed to login.(Ip=xxx.xx.xx,UserName=**,Times=1,AccessType=TELNET,VpnName=)',

[0082] 'dd':'01',

[0083] 'name':'SHELL',

[0084] 'severity':'4',

[0085] 'brief':'LOGINFAILED',

[0086] 'log_path':' / Users / xxxxxxxxx / fault_locate / A / test

[0087] data / CSG-8_xxx.xx.x.x'}.

[0088] That is, in each key-value pair of the present invention, the key can be the field name of the field that constitutes the log data, and the value can be the numerical value of the corresponding field.

[0089] After structuring the log data, the corresponding log events can be obtained from the structured log data.

[0090] It should be noted that, in this embodiment of the invention, a log event may include multiple fields, such as a host name field, an event field, and a module name field. That is, if E represents a log event, then a log event can be represented as: E = {w host ,w event ,…,w module}, where w host Indicates the hostname, w event To indicate an event, w module Indicates the module name.

[0091] For example, for the structured log data shown, the corresponding log events can be represented as E = {"CSG-8", "LOGINFAILED", ..., "SHELL"}.

[0092] Of course, this example uses a single log data entry. It is understood that in actual implementation, there are usually multiple log data entries to be analyzed. These multiple log data entries can be structured separately to obtain multiple log events.

[0093] Step S102: Concatenate multiple log events to obtain an event text sequence.

[0094] The concatenation of multiple log events includes: performing a preset sorting process on the multiple log events according to the log generation timestamp corresponding to each log event, and concatenating the sorted log events in order to obtain an event text sequence.

[0095] This preset sorting process can sort multiple log events in ascending order based on the log generation timestamp corresponding to the log event, where the log generation timestamp corresponding to the log event is used to indicate the generation time of the log event.

[0096] The log events are designated as E1, E2, ..., E n E1 = {w host-1 ,w event ,…,w module-1}, E2={w host- ,w event-2 ,…,w module-2},…,En ={w host- ,w event-n ,…,w module-n For example, in the case of}, where n is a positive integer, the log events can be sorted in ascending order based on the timestamp of the log data generated for each log event. If the log events in ascending order are E1, E2, ..., E n Then, the n log events can be concatenated in order to obtain the event text sequence shown below:

[0097] S = [CLS], w host- ,w event-1 ,…,[CLS],w host- ,w event-2 ,…,[CLS],…,w module-n .

[0098] Here, the symbol [CLS] is an abbreviation for "classification," which can be used to indicate the beginning of a sentence or document. In this embodiment of the invention, [CLS] is used to segment different log events in an event text sequence.

[0099] Step S103: Input the event text sequence into the target prediction model corresponding to the target prediction task for processing to obtain the target processing result of the log data on the target prediction task; wherein, the target prediction model includes an encoding model and a decoding model, and the encoding model is trained on a preset pre-trained language model using sample log data and log text vocabulary corresponding to the log data.

[0100] In this embodiment of the invention, the target prediction task can be any task corresponding to the text processing task.

[0101] The target prediction task can be, for example, an event prediction task, which refers to predicting at least one log event that a communication device may experience in the next moment based on one or more known log events.

[0102] The target prediction task can also be used to directly predict system failures of communication equipment. For example, if an event text sequence indicates that one or more modems in a nearby location are malfunctioning, the target prediction model can predict the log event "The communication equipment will experience a network failure".

[0103] Of course, this is just an example; in actual implementation, the target prediction task can also be other tasks, without special limitations here.

[0104] A target prediction model refers to a model corresponding to a target prediction task. This target prediction model can include an encoding model and a decoding model. The encoding model can be used to extract features from the input event text sequence to obtain the corresponding event feature sequence. The decoding model can be set according to the target prediction task to perform decoding processing based on the event feature sequence output by the encoding model and generate the target processing result.

[0105] For example, when the target prediction task is an event prediction task, the decoding model can be a generator model, which generates log events that the communication device may encounter in the next moment based on the event feature sequence output by the encoding model.

[0106] It should be noted that, in order for the target prediction model to accurately learn the structural features, temporal features, and other characteristics of the log data of the communication device, in this embodiment of the invention, the encoding model in the target prediction model can be a model obtained by training a preset pre-trained language model, such as Transformer or BERT, using sample log data corresponding to the log data and a log text vocabulary.

[0107] In this embodiment of the invention, the log text vocabulary, also known as a log dictionary, is a vocabulary containing commonly used words in the log data described in this embodiment. Based on this log text vocabulary, log data can be correctly segmented, avoiding the splitting of a complete token into multiple tokens with no business meaning. For example, based on this log text vocabulary, the complete token "CMDCONFIRM_CHANGEPWDCHOOSEINFO" can be completely extracted from the log data, instead of being split into multiple tokens "CMD", "CONFIRM", "CHANGE", "PWD", "CHOOSE", and "INFO" with no actual business meaning. This allows the target prediction model to accurately understand the structural characteristics of the log data, thereby improving the accuracy of the target processing results.

[0108] As can be seen, based on the method provided in this embodiment of the invention, log events corresponding to the log data are obtained by structuring the log data to be analyzed. Then, considering that there is often a temporal dependency between log events in communication devices, multiple log events in the log data are concatenated to obtain an event text sequence. This event text sequence is then input into the target prediction model corresponding to the target prediction task for processing. Since the encoding model included in the target prediction model is trained on a preset pre-trained language model using sample log data and a log text vocabulary corresponding to the log data, the target prediction model can analyze the log data while correctly extracting structural features, temporal features, and other domain features of the log data, thereby accurately obtaining the target processing result of the log data on the target prediction task.

[0109] In some embodiments, the target prediction task can be an event prediction task. In this embodiment, the encoding model can use log events as processing units, encode the features of each log event, and predict the event features of related log events based on the existing event features of the log events. For example, for log event E1, the encoding model can first encode its features to obtain its corresponding event feature e1. Then, the encoding model can predict the event features e2 of other log events that are temporally related to the event feature e1 based on the temporal dependencies between log events.

[0110] Specifically, in this embodiment, in step S103 above, the step of inputting the event text sequence into the target prediction model corresponding to the target prediction task for processing, and obtaining the target processing result of the log data on the target prediction task, includes:

[0111] Step S1031: Input the event text sequence into the encoding model for event encoding and event prediction processing to obtain the predicted event feature sequence; wherein, the predicted event feature sequence includes feature data of at least one predicted log event associated with the log event in the event text sequence;

[0112] Step S1032: Input the predicted event feature sequence into the decoding model of the target prediction model for log event generation processing to obtain at least one predicted log event;

[0113] Step S1033: Obtain the target processing result based on at least one predicted log event.

[0114] In this implementation, the decoding model of the target prediction model can be a generative model, which can be used to predict at least one predicted log event generated at a subsequent time or consecutive time based on the predicted event feature sequence output by the encoding model, and then obtain the target processing result based on the at least one predicted log event.

[0115] In some embodiments, after obtaining a predicted log event based on the method provided in this embodiment of the invention, the electronic device generates a prompt message corresponding to the alarm level Severity of the predicted log event, and pushes the prompt message to the terminal device used by the maintenance personnel to prompt the maintenance personnel to perform timely maintenance on the communication equipment.

[0116] As can be seen, based on the method provided in the embodiments of this disclosure, the event text sequence is predicted by the encoding model in the target prediction model. Since the event text sequence can reflect the structural characteristics of log data and the temporal characteristics between different log events, the feature data of at least one predicted log event associated with the log event in the event text sequence at the next moment, i.e. the first moment, can be accurately obtained by encoding and predicting the event text sequence based on the encoding model. The at least one predicted log event can be obtained by decoding the feature data, and then the target processing result is generated based on the at least one predicted log event.

[0117] Please refer to Figure 2 , Figure 3a , Figure 3b These are, respectively, a structural diagram of the encoding model provided in the embodiments of the present invention, a flowchart for training the encoding model, and a framework diagram for training the encoding model. Figure 2 As shown, the encoding model in this embodiment of the invention may include a text encoding sub-model θ. text (·) and event coding submodel θ event (·), the text encoding sub-model and the event encoding sub-model can be any model such as Transformer or BERT. The text encoding sub-model can be a pre-trained language model that processes text in units of words, and the event encoding sub-model can be a pre-trained language model that processes text in units of log events.

[0118] like Figure 3a , 3b As shown, this encoding model can be trained through the following steps S301-S302:

[0119] Step S301: The sample log data is structured and spliced ​​to obtain the sample event text sequence.

[0120] The sample log data can be data corresponding to the log data mentioned above. For example, the sample log data can be constructed by obtaining historical log data from communication devices.

[0121] In addition, the structuring and splicing of sample log data are explained in the relevant sections above, and will not be repeated here.

[0122] The sample event text sequence can be an event text sequence with a similar structure to the event text sequences described above. For ease of explanation, the following description will continue to use the sample event text sequence as an example.

[0123] S = [CLS], w host-1 ,w event-1 ,…,[CLS],w host- ,w event ,…,[CLS],…,w module-n

[0124] To clarify, each sample log event also includes multiple fields, and the log events in the sample event text sequence are separated by the symbol [CLS].

[0125] Step S302: Train the encoding model based on the sample event text sequence and log text vocabulary to obtain the encoding model.

[0126] For details regarding the log text vocabulary, please refer to the relevant sections mentioned above; they will not be repeated here.

[0127] In this embodiment of the invention, log data with the same or similar structure as the log data of the communication device can be obtained as sample log data. A sample event text sequence is constructed based on the sample log data. Then, based on the sample event text sequence and the log text vocabulary corresponding to the log data, the pre-trained language model is trained to obtain the encoding model.

[0128] It is understandable that the pre-trained language model can be a model initially trained using a large amount of natural language text as corpus, such as the Transformer or BERT model. In step S302, the pre-trained language model can be fine-tuned based on the sample event text sequence and the log text vocabulary to obtain an encoding model that can accurately acquire the structural features and temporal features of the log data described in this embodiment of the invention. After obtaining the encoding model, a corresponding decoding model can be added to the encoding model according to the target prediction task to be performed, thus constructing a target prediction model.

[0129] like Figure 3b As shown, in some embodiments, in step S202 above, training the encoding model based on the sample event text sequence and log text vocabulary to obtain the encoding model may include:

[0130] Step S2021: Perform word segmentation on the sample event text sequence based on the log text vocabulary.

[0131] The word segmentation process involves using the log text vocabulary to accurately split the words in the sample event text sequence; the detailed processing will not be elaborated here.

[0132] Step S2022: Mask at least one word of any log event in the sample event text sequence to obtain the first masked text sequence.

[0133] Step S2023: Input the sample event text sequence and the first occluded text sequence into the text encoding sub-model respectively to obtain the first text feature and the first occluded text feature;

[0134] Step S2024: Obtain the first loss of the encoding model based on the first text features and the first occluded text features;

[0135] Step S2025: Mask the sub-features in the first text feature that correspond to any log event to obtain the second text feature; and reorder the sub-features in the first text feature that correspond to each log event to obtain the third text feature.

[0136] Step S2026: Input the second text feature and the third text feature into the event coding sub-model respectively to obtain the first event feature and the second event feature. The first event feature is the event feature obtained by the event coding sub-model after predicting the occluded sub-features in the second text feature. The second event feature is the event feature obtained by the event coding sub-model after reordering the third text feature.

[0137] Step S2027: Obtain the second loss value based on the first text features, the first event features, and the second event features;

[0138] Step S2028: Adjust the parameters of the coding model according to the first loss value and the second loss value to obtain a coding model that meets the preset convergence conditions.

[0139] In step S2027, obtaining the second loss value based on the first text feature, the first event feature, and the second event feature includes: obtaining a third loss between the first text feature and the first event feature; and obtaining a fourth loss between the first text feature and the second event feature; and obtaining the second loss based on the third loss and the fourth loss.

[0140] In this implementation, steps S2021-S2024 correspond to the Masked Language Modeling (MLM) task. The specific idea behind the MLM task is to randomly mask at least one word from any log event in a sample event text sequence and input it into the text encoding sub-model θ. textThe hidden state identifier of the masked text sequence is obtained by encoding in (·), and the text encoding sub-model predicts it based on the context of the masked words, so that the model can learn the semantic relationship between words in the log event.

[0141] Specifically, in step S2022, at least one word of any log event in the sample event text sequence can be masked to obtain a first masked text sequence. In step S2023, the first masked text sequence and the sample event text sequence are respectively input into the text encoding sub-model for processing to obtain the first text feature corresponding to the unmasked sample event text sequence, and the first masked text feature after the text encoding sub-model predicts and fills the masked words according to the context information of the words. Then, in step S2024, a loss value can be obtained based on the error between the first text feature and the first masked text feature, i.e., the first loss value mentioned above, to adjust the parameters of the encoding model.

[0142] Let the sample text sequence be S=[CLS],w host-1 ,w event-1 ,…,[CLS],w host-2 ,w event-2 ,…,[CLS],…,w module-n For example, this can be achieved by analyzing the word "w" in the second log event. host-2 The text is masked to obtain the first masked text sequence S1 = [CLS],w host-1 ,w event-1 ,…,[CLS],Mask,w event-2 ,…,[CLS],…,w module-n Then, they can be input into the text encoding submodel θ respectively. text Encode it in (·) to obtain the first masked text feature, that is, the feature E containing the masked word. mask The text features, and the first text features containing the actual features corresponding to the masked words, can then be used to obtain a first loss based on the error between the first masked text features and the first text features.

[0143] In addition, the above steps S2025-S2028 correspond to the Event Prediction Modeling (EPM) task and the Event Ordering Recovering (EOR) task.

[0144] The event prediction simulation task is used to randomly mask any event in a sample event text sequence, so that the event encoding sub-model can predict the feature representation of the masked log event based on the feature representation of the log events surrounding the masked event. The third loss is obtained by calculating the error between the predicted feature representation of the masked log event and the true feature representation, and the parameters of the encoding model are adjusted based on the third loss.

[0145] The event sorting and recovery task is used to enable the encoding model to learn the temporal characteristics between log events in the log data. Specifically, this task involves shuffling the order of each log event in the sample text sequence, i.e., randomly reordering the log events. The reordered sample text sequence and the original sample text sequence are then input into the event encoding sub-model for encoding prediction. The event encoding sub-model reorders the reordered sample text sequence and outputs its feature representation. Then, based on the feature representation predicted by the model and the true feature representation of the sample text sequence, a fourth loss is obtained, and the parameters of the encoding model are adjusted based on this fourth loss.

[0146] Let the first text feature be E = {e1, e2, ..., e n For example, in the case of log event E1, e1 represents the characteristic of log event E1. n For log event E n The feature representation can be used to represent the sub-feature e corresponding to the i-th log event. i Masking is performed to obtain the second text feature, where 1≤i≤n. This second text feature is then input into the event encoding submodel θ. event (·) can be used to obtain the event sub-features after interaction, that is, the first event feature E′={e′1,e′2,…,e′ n From the first event feature, the predicted event sub-feature corresponding to the occluded i-th log event can be obtained, that is... Therefore, based on e i and The error between them is used to obtain a third loss, which is used to represent the prediction loss of the event-encoded sub-model.

[0147] For the task of sorting and recovering this event, the first text feature is still E = {e1, e2, ..., e}. n For example, the event sub-features can be reordered, that is, their order can be shuffled to obtain the corresponding third text feature E. shuff ={e m1 ,e m2 ,…,e mn}, E shuffle Input event encoding submodel θ event(·), the event encoding sub-model reorders the log events according to their temporal dependencies, thus obtaining the second event feature E′ of the predicted output. shuffe Then, based on the true order between the sub-features of each log event in the second event feature and the second text feature, the prediction loss of the event encoding sub-model can be calculated to obtain the fourth loss.

[0148] like Figure 3b As shown, in this embodiment of the invention, the second loss in step S2027 can be obtained by summing the third loss and the fourth loss, or the second loss can be obtained by weighted summing the third loss and the fourth loss according to the weight ratio of the EPM task and the EOR task in the training process of the encoding model. No special limitation is made here.

[0149] like Figure 3b As shown, in some embodiments, in step S2024 above, obtaining the first loss of the encoding model based on the first text feature and the first occluded text feature includes: inputting the first text feature and the first occluded text feature into the first perceptual network respectively to obtain the first sample score and the first occlusion score of the first text feature and the first occlusion score on the log text vocabulary; and obtaining the first loss based on the first sample score and the first occlusion score.

[0150] In practice, the first sensing network can be a multilayer sensor.

[0151] Specifically, the first text feature can be input into a multilayer perceptron, and then the softmax function can be used to obtain the probability distribution of the true text features of the sample text event sequence on the log text vocabulary as the first sample score; based on the same processing, the probability distribution of the first occluded text feature on the log text vocabulary can be obtained as the first occlusion score.

[0152] Let P represent the first masking score, and E represent the feature corresponding to the masked word. mask In this embodiment of the invention, the formula P = softmax(MLP(e) can be used. mask The first occlusion score is obtained.

[0153] After obtaining the first sample score and the first occlusion score, a loss function, such as the cross entropy loss function, can be used to obtain the first loss.

[0154] In some embodiments, obtaining the third loss between the first text features and the first event features includes: obtaining standard event features corresponding to the target log event from the first text features, wherein the target log event is the log event corresponding to the occluded sub-feature in the second text features; obtaining predicted event features corresponding to the target log event from the first event features; and obtaining the third loss based on the standard event features and the predicted event features.

[0155] For example, with e i Let represent the eclipsed i-th log event, i.e., the standard event characteristics corresponding to the target log event, and , with The third loss L represents the predicted event features obtained after predicting the target log event through the event coding sub-model. epm It can be calculated using the following formula:

[0156]

[0157] Of course, this is just an example. In actual implementation, the third loss can also be obtained through other means, and no special limitation is made here.

[0158] In some embodiments, obtaining the fourth loss between the first text feature and the second event feature includes: inputting the first text feature and the second event feature into a second perceptual network to obtain a first confidence sequence and a second confidence sequence, wherein the first confidence sequence is used to represent the confidence distribution of the log event sequence corresponding to the first text feature, and the second confidence sequence is used to represent the confidence distribution of the log event sequence represented by the second event feature; and obtaining the fourth loss based on the first confidence sequence and the second confidence sequence.

[0159] The second sensing network can be a multilayer perceptron.

[0160] In this implementation, the first text feature and the second event feature can be input into a multilayer perceptron, and then normalized using the softmax function. This yields a first confidence sequence Q representing the confidence distribution of the relative position of each log event in the original sequence, and a second confidence sequence O representing the confidence distribution of each log event at the current predicted relative position. Specifically, O and Q can be obtained using the following formulas:

[0161] O = softmax(MLP(θ) event (E shuffle )))=(o1,o2,…,o n );

[0162] Q = softmax(MLP(θ) event(E)))=(q1,q2,…,q n );

[0163] Since the goal is for the encoding model to learn the temporal dependencies between log events, the error between the first confidence sequence and the second confidence sequence can be obtained using KL divergence as the loss function, thus yielding the fourth loss L. eor That is, the fourth loss L eor It can be obtained through the following formula:

[0164]

[0165] In practice, the sum of the first and second losses can be used as the target loss of the encoding model in a training round. The parameters of the encoding model can be adjusted based on this target loss to obtain an encoding model that can accurately understand the structural and temporal characteristics of log data. By constructing a target prediction model based on this encoding model, the accuracy of the target prediction model in analyzing and processing log data can be improved.

[0166] As described above, in this embodiment of the invention, during the training of the encoding model in the target prediction model for analyzing and processing log data of communication devices, considering the structural characteristics of the log data, an EPM task is designed during the training of the encoding model. By encoding the log data into log event-level feature representations and by narrowing the similarity between the predicted event features of randomly masked log events (i.e., the first event feature) and the original event features corresponding to the log event, the encoding model can accurately learn the structural features of the log data at the event level, thereby accurately encoding and representing the log data. Furthermore, based on the EOR task designed during the training of the encoding model, by comparing the first text feature representing the original sequence features of each log event with the second event feature obtained after predicting the sequence of the reordered log events, the encoding model can also accurately learn the temporal dependency information of each log event in the log data, thus providing a basis for accurately analyzing and processing the log data in downstream tasks.

[0167] It should be noted that the above explanation uses the target prediction task as an example of the event prediction task. This target prediction task can also be used to predict system faults of communication equipment. In this implementation, the decoding model in the target prediction model can be a classifier or a fully connected layer. The decoding model can predict the system faults of the communication equipment based on the second event feature sequence output by the encoding model and generate alarm information.

[0168] For example, if the second event feature sequence includes a log event of modem failure, the target prediction model can predict a possible "network outage" failure and generate corresponding alarm information so that maintenance personnel can maintain the status of communication equipment in a timely manner based on the alarm information.

[0169] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0170] This invention also provides a log analysis device, such as... Figure 4 As shown, the log analysis device 400 includes: a structured processing module 401, an event splicing module 402, and an analysis module 403.

[0171] The structured processing module 401 is used to perform structured processing on the log data to be analyzed, and obtain log events corresponding to the log data.

[0172] The event concatenation module 402 is used to concatenate multiple log events to obtain an event text sequence.

[0173] The analysis module 403 is used to input the event text sequence into the target prediction model corresponding to the target prediction task for processing, and to obtain the target processing result of the log data on the target prediction task; wherein, the target prediction model includes an encoding model and a decoding model, and the encoding model is obtained by training a preset pre-trained language model using sample log data and a log text vocabulary corresponding to the log data.

[0174] In some embodiments, the target prediction task includes an event prediction task; when the analysis module 403 processes the event text sequence by inputting it into a target prediction model corresponding to the target prediction task to obtain the target processing result of the log data on the target prediction task, it can be used to: input the event text sequence into the encoding model for event encoding and event prediction processing to obtain a predicted event feature sequence; wherein, the predicted event feature sequence includes feature data of at least one predicted log event associated with the log event in the event text sequence; input the predicted event feature sequence into the decoding model of the target prediction model for log event generation processing to obtain the at least one predicted log event; and obtain the target processing result based on the at least one predicted log event.

[0175] In some embodiments, the device further includes a training module, which can be used to: perform structuring and concatenation processing on the sample log data to obtain a sample event text sequence; and train the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model.

[0176] In some embodiments, the encoding model includes a text encoding sub-model and an event encoding sub-model. When the training module trains the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model, it can be used to: perform word segmentation on the sample event text sequence based on the log text vocabulary; mask at least one word of any log event in the sample event text sequence to obtain a first masked text sequence; input the sample event text sequence and the first masked text sequence into the text encoding sub-model respectively to obtain a first text feature and a first masked text feature; obtain a first loss of the encoding model based on the first text feature and the first masked text feature; mask the sub-features in the first text feature corresponding to any log event, obtaining... The second text feature is obtained, and the sub-features corresponding to each log event in the first text feature are reordered to obtain the third text feature; the second text feature and the third text feature are respectively input into the event encoding sub-model to obtain the first event feature and the second event feature, wherein the first event feature is the feature obtained by the event encoding sub-model after predicting the masked sub-features in the second text feature, and the second event feature is the feature obtained by the event encoding sub-model after reordering the sub-features in the third text feature; a second loss value is obtained based on the first text feature, the first event feature and the second event feature; the parameters of the encoding model are adjusted based on the first loss value and the second loss value to obtain the encoding model that satisfies the preset convergence condition.

[0177] In some embodiments, when the training module obtains a second loss value based on the first text feature, the first event feature, and the second event feature, it can be used to: obtain a third loss between the first text feature and the first event feature; and obtain a fourth loss between the first text feature and the second event feature; and obtain the second loss based on the third loss and the fourth loss.

[0178] In some embodiments, when the training module obtains the first loss of the encoding model based on the first text feature and the first occluded text feature, it can be used to: input the first text feature and the first occluded text feature into a first perceptual network respectively to obtain a first sample score and a first occlusion score of the first text feature and the first occlusion score on the log text vocabulary; and obtain the first loss based on the first sample score and the first occlusion score.

[0179] In some embodiments, when the training module obtains the third loss between the first text features and the first event features, it can be used to: obtain standard event features corresponding to the target log event from the first text features, wherein the target log event is the log event corresponding to the occluded sub-feature in the second text features; obtain predicted event features corresponding to the target log event from the first event features; and obtain the third loss based on the standard event features and the predicted event features.

[0180] In some embodiments, when the training module obtains the fourth loss between the first text feature and the second event feature, it can be used to: input the first text feature and the second event feature into the second perceptual network respectively to obtain a first confidence sequence and a second confidence sequence, wherein the first confidence sequence is used to represent the confidence distribution of the log event sequence corresponding to the first text feature, and the second confidence sequence is used to represent the confidence distribution of the log event sequence represented by the second event feature; and obtain the fourth loss based on the first confidence sequence and the second confidence sequence.

[0181] The apparatus provided in the embodiments of the present invention has functions or includes modules that can be used to perform the methods described in the first aspect of the method embodiments above. The specific implementation and technical effects can be referred to the description of the method embodiments above. For the sake of brevity, they will not be repeated here.

[0182] It should be noted that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0183] Reference Figure 5 This invention provides an electronic device comprising:

[0184] One or more processors 501;

[0185] The memory 502 stores one or more programs that, when executed by one or more processors, enable the one or more processors to implement any of the above-mentioned log analysis methods.

[0186] One or more I / O interfaces 503 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0187] Among them, processor 501 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 503 is connected between processor 501 and memory 502, and can realize information interaction between processor 501 and memory 502, including but not limited to data bus (Bus).

[0188] In some embodiments, the processor 501, memory 502, and I / O interface 503 are interconnected via a bus, and thus connected to other components of the computing device.

[0189] This embodiment also provides a computer-readable medium having a computer program stored thereon. When the program is executed by a processor, it implements the log analysis method provided in this embodiment. To avoid repetition, the specific steps of the log analysis method will not be repeated here.

[0190] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses in the methods, systems, and apparatuses described above can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0191] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0192] Those skilled in the art will understand that although some embodiments described herein include certain features that are included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this embodiment and form different embodiments.

[0193] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A log analysis method, characterized in that, include: The log data to be analyzed is structured to obtain the log events corresponding to the log data; Concatenate multiple log events to obtain an event text sequence; The event text sequence is input into the target prediction model corresponding to the target prediction task for processing, and the target processing result of the log data on the target prediction task is obtained. The target prediction model includes an encoding model and a decoding model. The encoding model is obtained by training a pre-trained language model using sample log data and a log text vocabulary corresponding to the log data. The encoding model includes a text encoding sub-model and an event encoding sub-model. The encoding model is trained through the following steps: structuring and concatenating the sample log data to obtain a sample event text sequence; and training the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model. The step of training the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model includes: The sample event text sequence is segmented based on the log text vocabulary. At least one word of any log event in the sample event text sequence is masked to obtain a first masked text sequence; The sample event text sequence and the first occluded text sequence are respectively input into the text encoding sub-model to obtain the first text feature and the first occluded text feature; Based on the first text feature and the first masked text feature, obtain the first loss value of the encoding model; Masking the sub-features in the first text feature corresponding to any log event to obtain the second text feature; and reordering the sub-features in the first text feature corresponding to each log event to obtain the third text feature. The second text feature and the third text feature are respectively input into the event encoding sub-model to obtain the first event feature and the second event feature. The first event feature is the feature obtained by the event encoding sub-model after predicting the occluded sub-features in the second text feature, and the second event feature is the feature obtained by the event encoding sub-model after reordering the sub-features in the third text feature. Based on the first text features, the first event features, and the second event features, a second loss value is obtained; Based on the first loss value and the second loss value, the parameters of the encoding model are adjusted to obtain the encoding model that satisfies the preset convergence condition.

2. The method according to claim 1, characterized in that, The target prediction task includes an event prediction task; The step of inputting the event text sequence into the target prediction model corresponding to the target prediction task for processing, and obtaining the target processing result of the log data on the target prediction task, includes: The event text sequence is input into the encoding model for event encoding and event prediction processing to obtain a predicted event feature sequence; wherein, the predicted event feature sequence includes feature data of at least one predicted log event associated with the log event in the event text sequence; The predicted event feature sequence is input into the decoding model of the target prediction model for log event generation processing to obtain the at least one predicted log event; The target processing result is obtained based on the at least one predicted log event.

3. The method according to claim 1, characterized in that, The step of obtaining the second loss value based on the first text feature, the first event feature, and the second event feature includes: Obtain a third loss between the first text feature and the first event feature; and obtain a fourth loss between the first text feature and the second event feature; The second loss value is obtained based on the third loss and the fourth loss.

4. The method according to claim 1, characterized in that, The step of obtaining the first loss value of the encoding model based on the first text feature and the first masked text feature includes: The first text feature and the first occluded text feature are respectively input into the first perception network to obtain the first sample score and the first occlusion score of the first text feature and the first occlusion text feature on the log text vocabulary; The first loss value is obtained based on the first sample score and the first occlusion score.

5. The method according to claim 3, characterized in that, The step of obtaining the third loss between the first text features and the first event features includes: From the first text features, obtain the standard event features corresponding to the target log event, wherein the target log event is the log event corresponding to the occluded sub-feature in the second text features; Obtain the predicted event features corresponding to the target log event from the first event features; The third loss is obtained based on the standard event characteristics and the predicted event characteristics.

6. The method according to claim 3, characterized in that, The step of obtaining the fourth loss between the first text features and the second event features includes: The first text feature and the second event feature are respectively input into the second perceptual network to obtain a first confidence sequence and a second confidence sequence, wherein the first confidence sequence is used to represent the confidence distribution of the log event sequence corresponding to the first text feature, and the second confidence sequence is used to represent the confidence distribution of the log event sequence represented by the second event feature. The fourth loss is obtained based on the first confidence sequence and the second confidence sequence.

7. A log analysis device, characterized in that, include: The structured processing module is used to perform structured processing on the log data to be analyzed, and obtain the log events corresponding to the log data. The event concatenation module is used to concatenate multiple log events to obtain an event text sequence; The analysis module is used to input the event text sequence into the target prediction model corresponding to the target prediction task for processing, and to obtain the target processing result of the log data on the target prediction task; The target prediction model includes an encoding model and a decoding model. The encoding model is obtained by training a pre-trained language model using sample log data and a log text vocabulary corresponding to the log data. The encoding model includes a text encoding sub-model and an event encoding sub-model. The encoding model is trained through the following steps: structuring and concatenating the sample log data to obtain a sample event text sequence; and training the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model. The step of training the encoding model based on the sample event text sequence and the log text vocabulary to obtain the encoding model includes: The sample event text sequence is segmented based on the log text vocabulary. At least one word of any log event in the sample event text sequence is masked to obtain a first masked text sequence; The sample event text sequence and the first occluded text sequence are respectively input into the text encoding sub-model to obtain the first text feature and the first occluded text feature; Based on the first text feature and the first masked text feature, obtain the first loss value of the encoding model; Masking the sub-features in the first text feature corresponding to any log event to obtain the second text feature; and reordering the sub-features in the first text feature corresponding to each log event to obtain the third text feature. The second text feature and the third text feature are respectively input into the event encoding sub-model to obtain the first event feature and the second event feature. The first event feature is the feature obtained by the event encoding sub-model after predicting the occluded sub-features in the second text feature, and the second event feature is the feature obtained by the event encoding sub-model after reordering the sub-features in the third text feature. Based on the first text features, the first event features, and the second event features, a second loss value is obtained; Based on the first loss value and the second loss value, the parameters of the encoding model are adjusted to obtain the encoding model that satisfies the preset convergence condition.

8. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method according to any one of claims 1-6; One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

9. A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1-6.

Images