Log anomaly detection method and device, computer device and storage medium

Abstract

The application discloses a log anomaly detection method and device, computer equipment and a storage medium, relates to the technical field of log processing, and comprises the following steps: determining the initial word sequence of each target log in a target log sequence; performing vectorization on each initial word sequence in the target log sequence through a pre-trained SBERT submodel, determining a log parameter matrix of the target log sequence and a log template feature matrix containing sequence context information in the target log sequence through a feature extraction submodel, performing sequence-level anomaly detection and single log anomaly detection on the target log sequence in parallel through a classification task submodel, and determining the final anomaly detection result of the target log sequence based on the anomaly detection results of two different levels. Through the parallel synergy of different granularity detection tasks, the model can comprehensively identify anomalies, the generalization ability and detection precision of the model are improved, and the anomaly detection precision for complex and variable log sequences is ensured.

Description

TECHNICAL FIELD

[0001] The present disclosure generally relates to the technical field of log processing, and particularly relates to a log anomaly detection method and device, a computer device and a storage medium. BACKGROUND

[0002] Logs, as historical records generated by system software during the running process, are one of the core bases for developers, operation and maintenance personnel and security personnel to debug and maintain system software. In addition, anomaly detection of logs can capture potential risks in system software, discover abnormal events in time, and provide warning information for relevant personnel. Therefore, how to accurately and quickly detect whether logs are abnormal is particularly important.

[0003] In related technologies, a binary classifier is usually used to solve the log anomaly detection task. First, a word embedding technology is used to vectorize the log template, then a Bi-directional Long Short-Term Memory (BiLSTM) network and an attention mechanism are used to generate a single vector representing the log sequence, and finally a binary classifier is used to determine whether the log is abnormal.

[0004] However, since the existing log anomaly detection method mainly focuses on the pattern error of the log sequence, and only uses the log template as the basis for determining whether the log sequence is abnormal, although some methods also consider the log parameters, the log template and the log parameters are usually modeled separately, and feature fusion is performed inside the model. Therefore, for complex and variable log sequences, the existing log anomaly detection method has certain limitations. SUMMARY

[0005] In view of the above defects or deficiencies in the prior art, it is desirable to provide a log anomaly detection method, device, computer device and storage medium, which can not only use a pre-trained SBERT model to avoid the assumption of model closure and stability of log patterns, but also design different granularity detection tasks to detect anomalies in parallel from two granularities of log sequences and individual logs. The sequence detection task only uses template information to avoid noise introduced by dynamic parameters. The individual log detection task combines template and parameter information to retain the complete features of log information. The two detection tasks share the feature extraction sub-model of the model. The synergistic effect of the double tasks enables the model to more comprehensively identify anomalies, improves the generalization ability and detection accuracy of the model, and thus ensures the anomaly detection accuracy for complex and variable log sequences.

[0006] In a first aspect, the present application provides a log anomaly detection method. The method comprises:

[0007] determining an initial token sequence of each target log in a target log sequence;

[0008] input each of the initial token sequences into a pre-training SBERT sub-model in the log anomaly detection model, vectorize each of the initial token sequences to obtain an initial token vector matrix corresponding to each of the initial token sequences;

[0009] input each of the initial token vector matrices into a feature extraction sub-model in the log anomaly detection model, determine a log parameter matrix and a log template matrix of the target log sequence from the plurality of initial token vector matrices, and extract a context dependency relationship inside the target log sequence based on the log template matrix and position information of the log template matrix to obtain a log template feature matrix containing sequence context information in the target log sequence;

[0010] input the log template feature matrix, the target log sequence, and the log parameter matrix into a classification task sub-model in the log anomaly detection model, perform overall anomaly detection on the target log sequence based on the log template feature matrix, while performing local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix, the local anomaly detection adopts a determination strategy combining a sequence length-based dynamic threshold function and a voting mechanism; determine the final anomaly detection result of the target log sequence based on the overall anomaly detection result and the local anomaly detection result.

[0011] In combination with the first aspect, in a possible implementation manner, the overall anomaly detection on the target log sequence based on the log template feature matrix comprises:

[0012] determine an attention weight vector of the target log sequence based on a query vector and the log template feature matrix;

[0013] perform Softmax normalization processing on the attention weight vector, and aggregate all column vectors in the log template feature matrix based on the normalized attention weight vector to obtain a sequence feature column vector of the target log sequence, the sequence feature column vector being used to represent overall sequence information of the target log sequence;

[0014] perform feature extraction and merging on the sequence feature column vector according to full connection layer parameters to obtain a sequence anomaly real number representing overall anomaly of the target log sequence;

[0015] perform mapping of the sequence anomaly real number in a preset numerical range, and the mapping result is a sequence anomaly probability of the target log sequence and belongs to the overall anomaly detection result.

[0016] In a possible implementation manner of the first aspect, the local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix comprises:

[0017] splicing the log template feature vector corresponding to the target log in the log template feature matrix with the log parameter vector corresponding to the target log in the log parameter matrix to obtain a log feature splicing vector of the target log;

[0018] performing feature extraction and merging on the log feature splicing vector according to the full connection layer parameters to obtain a log anomaly real number representing an abnormal situation of the target log;

[0019] mapping the log anomaly real number to a preset numerical range, and a mapping result is a log anomaly probability of the target log and belongs to the local anomaly detection result.

[0020] In a possible implementation manner of the first aspect, the method further comprises:

[0021] in a case where the local anomaly detection result comprises a log anomaly probability of each target log in the target log sequence, determining an abnormal log in the target log sequence based on a size relationship between each log anomaly probability and a preset log anomaly probability threshold value, wherein the preset log anomaly probability threshold value is determined based on the dynamic threshold function;

[0022] determining a first anomaly detection result of the target log sequence based on a size relationship between a proportion of the abnormal log in the target log sequence and a preset proportion threshold value, wherein the process of determining the first anomaly detection result is determined by the voting mechanism;

[0023] in a case where the global anomaly detection result comprises the sequence anomaly probability, determining a second anomaly detection result of the target log sequence based on a size relationship between the sequence anomaly probability and a preset sequence probability threshold value;

[0024] performing a logical OR operation on the first anomaly detection result and the second anomaly detection result to obtain the final anomaly detection result.

[0025] In a possible implementation manner of the first aspect, the method further comprises:

[0026] in a case where each Transformer encoder in the pre-trained SBERT submodel comprises a multi-head self-attention mechanism module and a multi-layer perception module, and the classification task submodel comprises an attention pooling module and a classifier module,

[0027] in response to the software version updating instruction, inserting each preset MLP-Adapter module into the corresponding Transformer encoder in parallel with the corresponding multi-layer perception module, and inserting each preset ATT-Adapter module after the corresponding multi-head self-attention mechanism module and in series with the multi-head self-attention mechanism module, to obtain an updated SBERT model;

[0028] replacing the pre-trained SBERT model with the updated SBERT model to obtain an updated log anomaly detection model;

[0029] retraining the updated log anomaly detection model using a sample log sequence; the retraining includes parameter update training of module parameters of each preset MLP-Adapter module, each preset ATT-Adapter module, the attention pooling module, and the classifier module, and a parameter matrix in the preset ATT-Adapter module uses a low-rank matrix of a preset dimension; the retrained log anomaly detection model is used for log anomaly detection on a new target log sequence.

[0030] In combination with the first aspect, in a possible implementation manner, the determining of the log parameter matrix and the log template matrix of the target log sequence from the plurality of initial token vector matrices includes:

[0031] decomposing each of the initial token vector matrices into an initial token template matrix and an initial token parameter matrix;

[0032] performing vector aggregation on a group of initial token template vectors contained in each of the initial token template matrices and a group of initial token parameter vectors contained in each of the initial token parameter matrices, respectively, to obtain the log parameter matrix and the log template matrix.

[0033] In combination with the first aspect, in a possible implementation manner, the determining of the initial token sequence of each target log in the target log sequence includes:

[0034] performing preprocessing on an original log sequence collected from a target application software to obtain the target log sequence; wherein the preprocessing includes deleting empty logs and logs containing missing fields in the original log sequence, space replacement of special characters in the logs, merging adjacent continuous spaces, and standardization processing of content formats in the logs;

[0035] performing token segmentation on each target log in the target log sequence to obtain the initial token sequence of each target log.

[0036] In a second aspect, the present application also provides a log anomaly detection device. The device comprises:

[0037] a word sequence determination unit configured to determine an initial word sequence of each target log in the target log sequence;

[0038] a log feature extraction unit configured to input each of the initial word sequences into a pre-trained SBERT sub-model in a log anomaly detection model, vectorize each of the initial word sequences to obtain an initial word vector matrix corresponding to each of the initial word sequences, input each of the initial word vector matrices into a feature extraction sub-model in the log anomaly detection model, determine a log parameter matrix and a log template matrix of the target log sequence from the plurality of initial word vector matrices, and extract a context dependency relationship inside the target log sequence based on the log template matrix and position information of the log template matrix to obtain a log template feature matrix containing sequence context information in the target log sequence;

[0039] a log anomaly detection unit configured to input the log template feature matrix, the target log sequence, and the log parameter matrix into a classification task sub-model in the log anomaly detection model, perform overall anomaly detection on the target log sequence based on the log template feature matrix, at the same time, perform local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix, the local anomaly detection adopting a determination strategy of a dynamic threshold function based on sequence length combined with a voting mechanism, and determine a final anomaly detection result of the target log sequence based on an overall anomaly detection result and a local anomaly detection result.

[0040] In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the log anomaly detection method of the first aspect when executing the computer program.

[0041] In a fourth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the log anomaly detection method of the first aspect.

[0042] The application provides a log anomaly detection method, device, computer equipment and storage medium, wherein the log anomaly detection method comprises the following steps: performing vectorization on an initial token sequence of each target log in a target log sequence through a pre-trained SBERT submodel; determining a log parameter matrix of the target log sequence and a log template feature matrix containing sequence context information in the target log sequence based on an initial token vector matrix corresponding to each initial token sequence through a feature extraction submodel; and performing parallel sequence-level anomaly detection and single log anomaly detection on the target log sequence through a classification task submodel, and determining a final anomaly detection result of the target log sequence based on anomaly detection results of two different levels. In this way, the use of the pre-trained SBERT model can avoid the assumption that the model is closed and stable for log patterns, and by designing detection tasks of different granularities, parallel anomaly detection is performed from two granularities of log sequences and single logs, respectively. The sequence detection task only uses template information to avoid noise introduced by dynamic parameters; the single log detection task combines template and parameter information to retain complete features of log information, and the two detection tasks share the feature extraction submodel of the model and mutually serve as regular terms for the synergistic effect of double tasks, so that the model can more comprehensively identify anomalies, improve the generalization ability and detection precision of the model, and thus ensure the anomaly detection precision for complex and variable log sequences. BRIEF DESCRIPTION OF DRAWINGS

[0043] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings:

[0044] Figure 1 It is a flowchart of a log anomaly detection method in one embodiment;

[0045] Figure 2 It is a flowchart of a log anomaly detection method in one embodiment;

[0046] Figure 3 It is a log preprocessing and analysis diagram in one embodiment;

[0047] Figure 4 It is a flowchart of a log anomaly detection method in one embodiment;

[0048] Figure 5 It is a flowchart of a log anomaly detection method in one embodiment;

[0049] Figure 6 It is a flowchart of a log anomaly detection method in one embodiment;

[0050] Figure 7 It is a flowchart of a log anomaly detection method in one embodiment;

[0051] Figure 8 Fig. 1 is a schematic diagram of the insertion position and structure of two different Adapter modules in one embodiment;

[0052] Figure 9 Fig. 2 is a schematic diagram of the multi-task learning framework structure in one embodiment;

[0053] Figure 10 Fig. 3 is a structural block diagram of the log anomaly detection device in one embodiment;

[0054] Figure 11 Fig. 4 is an internal structure diagram of the computer device in one embodiment. DETAILED DESCRIPTION

[0055] The application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0056] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments. In addition, the term "and / or" in this paper is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. The terms "first" and "second" in the specification and claims of the embodiments of the present application are used to distinguish different objects, and not to describe the specific order of the objects.

[0057] Logs, as the historical records generated by system software during its operation, are one of the core references for developers, operators, and security personnel to debug and maintain system software. Log files are often referred to as the "black box" of system software. By embedding log output statements in the code, developers can capture key data such as the execution process, variable state, and exception information of the program. When the system software enters an abnormal state, the relevant process will automatically record the information and write it into the log file. These log files not only help developers to troubleshoot and reproduce problems, but also provide key data support for pattern recognition and prediction of system abnormal behavior. Based on logs, anomaly detection can capture potential risks in system software, detect abnormal events in a timely manner, and provide warning information to relevant personnel. Through automated analysis of a large amount of log data, abnormal behavior of the system can be quickly identified, helping operators to efficiently locate problems and reducing the time and cost of human intervention. In addition, log data can also support the health status evaluation, performance optimization, and security protection of the system. The value of log anomaly detection technology in fault diagnosis, risk prediction, and system optimization has become a focus of attention in the industry and academia. Therefore, developing efficient and intelligent log anomaly detection methods has important practical significance and broad application prospects.

[0058] With the development of deep learning and pre-trained language models in recent years, some research has begun to introduce them into log anomaly detection. For example, DeepLog models log templates, uses a recurrent neural network (RNN) to capture template sequence information, learns the patterns of normal log template sequences, and determines whether a log sequence is abnormal by outputting the probability distribution of the next log template and determining whether the true value exists in the Top-N probability options. LogBERT uses an unsupervised training mode, uses mask prediction and hypersphere minimization tasks in the pre-training phase to learn the patterns of normal template sequences, and uses a similar method as DeepLog in the prediction phase. LogRobust models log anomaly detection as a classic binary classification problem, first vectorizes log templates through word embedding technology, then uses BiLSTM network and Attention mechanism to generate a single vector representing the log sequence, and determines whether the log is abnormal through a binary classifier.

[0059] It can be understood that most of the existing log anomaly detection methods mainly use log templates, and assume that log templates have closed and stable characteristics. Log templates can capture the structured patterns of logs, help the model identify common software behavior patterns, and the parameter part may contain timestamps, error levels, and other rich information, which is crucial for identifying specific types of anomalies. For log sequence pattern detection, the template part may dominate. Log data may introduce errors during collection and preprocessing, and more seriously, even the most advanced log parser may inevitably introduce noise due to the distribution of data in the training set, i.e. errors will identify parameters as templates, or identify template parts as parameters and discard, the lack of information may lead to misjudgment and reduce the accuracy of the model. For single log instance detection, parameters also provide important information. Therefore, using only templates as the basis for determining whether a log sequence is abnormal not only may miss critical information, but also may cause the model to perform poorly when facing single log instance anomalies. Although some existing methods also consider templates and parameters, these methods split logs into templates and parameters and then send them into embedding models, which artificially breaks the context continuity, causing the model to fail to capture the potential semantic dependencies between templates and parameters. At the same time, if the parser returns an incorrect result, the parsing error will directly pollute the embedding input, affecting the results and making them irreparable. These methods fuse the extracted template and parameter features within the model, making it difficult for the model to weigh the importance of the two sources. The extracted feature expression may be chaotic, and the model still outputs a single result, unable to provide fine-grained feedback on whether the log sequence as a whole is abnormal or a single log is abnormal.

[0060] From the perspective of software iteration, log patterns usually change smoothly, i.e. the difference in log patterns between versions is not too large. In addition, different software systems may have similarities in describing similar problems. Therefore, the model should have the ability to retain knowledge and continuously learn. Existing methods do not fully consider this issue, and usually use full parameter update for the model, i.e. each version of the software needs to retrain the model and update all parameters, and for different versions of the software, all parameters of the corresponding model need to be saved for subsequent use. This approach not only consumes a large amount of computing power, but also causes storage space to expand, increasing maintenance costs and limiting its scalability and practicality in actual applications.

[0061] To solve the above technical problems, the application provides a log anomaly detection method, a computer device and a storage medium.(1) The complete log text is input into the SBERT model as a whole, the parameter and template information are distinguished at the token level in combination with the log parsing result, and the vector representation is extracted by performing mean pooling respectively, so that the semantic context continuity is maintained and the fine modeling of the structure information is realized.Compared with the prior art method, the prior art method avoids the context fragmentation, enhances the capturing ability of the model to the potential semantic dependence between the parameters and the templates, is more fault-tolerant to the inaccurate logs, and enhances the stability and robustness of the model.(2) In view of the two types of requirements of "local anomaly" and "sequence anomaly" in log anomaly detection, the application first designs a single log detection task and a log sequence detection task in the log anomaly detection multi-task learning mechanism, and adopts differentiated input feature design: the former jointly uses the parameter and template vectors to retain complete semantic features, and the latter only uses the template vector with context information to suppress the influence of parameter noise.The double-task shared feature extraction sub-module realizes structure orthogonality and semantic complementarity through joint training, improves the breadth and generalization ability of detection.(3) In the reasoning stage, the application proposes a single log judgment threshold function based on dynamic adjustment of sequence length to solve the problem of false positive accumulation when the sequence length increases, and combines the abnormal log proportion to control the sequence judgment threshold, which forms a double-task joint decision mechanism with the sequence detection output result, effectively alleviates the false alarm accumulation, and enhances the stability and fault tolerance of the model in actual deployment.(4) In view of the high parameter updating and storage overhead of the model in version migration and continuous updating, the application proposes a double-Adapter mode (parallel MLP-Adapter and serial ATT-Adapter), the main model parameters remain frozen, and only the Adapter module and the task head are updated, which has the characteristics of modularity and pluggability. Figures 1 to 11 The log anomaly detection method, device, computer device and storage medium of the application are described below, wherein the execution subject of the log anomaly detection method can be a computer device, which can be a personal computer, a server, an embedded system or other devices. The present application does not make specific limitations. Further, the log anomaly detection method can also be applied to the log anomaly detection device provided in the computer device, which can be realized by software, hardware or a combination of both. The log anomaly detection method is described below with the execution subject of the log anomaly detection method being a computer device as an example.

[0062] In order to facilitate understanding of the log anomaly detection method provided by the embodiments of the application, the log anomaly detection method provided by the application will be described in detail below through the following several example embodiments. It can be understood that the following several example embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments.

[0063] Referring to Figure 1 , one of flowcharts of the log anomaly detection method provided by the embodiments of the present application is shown in FIG. 1, which includes the following steps 101-104. Figure 1

[0064] Step 101, determining an initial token sequence of each target log in a target log sequence.

[0065] The target log sequence is a log sequence without empty logs, and each target log is a log without missing fields, special characters, continuous spaces, and content format standardization. Each target log can include a timestamp, an event type, a log level, and a message content, etc.

[0066] Each initial token sequence is a fine-grained token sequence representation of the corresponding target log, and each token is defined as a natural language unit (e.g., a word, a character, a number, etc.).

[0067] Specifically, the computer device determines the target log sequence from a large number of log instances stored by itself, which meets the requirements of the target log, and then all the filtered logs are used as target logs in the target log sequence. The determination process of the target log sequence is not limited here.

[0068] In addition, for the target log sequence, the computer device can determine the initial token sequence of each target log using a token generation algorithm, and the specific process of determining the initial token sequence of each target log is not limited by the present application.

[0069] Step 102, inputting each initial token sequence into a pre-trained SBERT sub-model in the log anomaly detection model, vectorizing each initial token sequence to obtain an initial token vector matrix corresponding to each initial token sequence.

[0070] The pre-trained SBERT sub-model can include N stacked pre-trained Transformer encoders (T-Encoders), where N is a positive integer greater than 0. That is, the pre-trained SBERT sub-model is a pre-trained language model based on the Transformer architecture.

[0071] The full name of SBERT in English is Sentence-Bidirectional Encoder Representations from Transformers, and the Chinese full name can be Sentence Embedding-Bidirectional Encoder Representation from Transformers, Sentence Embedding-Bidirectional Transformer Model, or Sentence Embedding-Bidirectional Encoder Representation from Transformer. ​

[0072] Specifically, by inputting each initial token sequence into the pre-trained SBERT sub-model, the output of the last hidden layer of the pre-trained SBERT sub-model before aggregation is obtained; here, the output is the vector representation corresponding to each token in each initial token sequence, thereby obtaining an initial token vector matrix corresponding to each initial token sequence.

[0073] It should be noted that although the default output of the pre-trained SBERT sub-model will aggregate the vector corresponding to each token to obtain the vector representation of a single target log, in order to more finely utilize the semantic information corresponding to the template and the parameter, the present application selects the output of the last hidden layer of the pre-trained SBERT sub-model before aggregation to obtain the vector representation corresponding to each token, which not only avoids the semantic fragmentation caused by log disassembly, but also enables the template and the parameter to be represented in the same semantic space and closely combined with the multi-task learning structure.

[0074] For example, let be a real field , and let be a set of all shapes of matrices, and let any one of the target logs in the target log sequence be taken as an example, the pre-trained SBERT sub-model outputs the token vector of all tokens corresponding to the target log, denoted as an initial token vector matrix , where h denotes the number of tokens of the target log, and the column vector denotes the vector representation of the h token in the k token, d denotes the dimension of the vector.

[0075] Step 103, input each initial token vector matrix into the feature extraction sub-model in the log anomaly detection model, determine the log parameter matrix and the log template matrix of the target log sequence from the plurality of initial token vector matrices, and extract the context dependency relationship inside the target log sequence based on the position information of the log template matrix and the log template matrix, to obtain a log template feature matrix containing sequence context information in the target log sequence.

[0076] Specifically, by inputting each initial token vector matrix into the feature extraction sub-model, the token-level vector of the log template and the token-level vector of the log parameter in each initial token vector matrix can be first subjected to an average pooling operation, aggregating the tokens of each target log to generate a log parameter vector and a log template vector of each target log, respectively, and then obtaining the log parameter matrix and the log template matrix of the target log sequence.

[0077] To introduce the sequence position information, the log template vectors are position information encoded using the sine-cosine position encoding, and the log template vector corresponding to each target log in the template log sequence is input into two layers of T-Encoders respectively to capture the context dependency relationship inside the target log sequence. Through the stacking of multiple layers of T-Encoders, the feature extraction sub-model can effectively capture the long-distance dependency relationship in the target log sequence and enhance the expression ability of the features.

[0078] For example, when the target log sequence includes n target logs, the log template matrix of the target log sequence is and the log parameter matrix is , and the feature extraction sub-model includes two layers of T-Encoders, and each layer of T-Encoders includes a Multi Head Self-Attention Mechanism (MHSA) module, a Multilayer Perceptron (MLP) module, a Residual Connection module and a Layer Normalization module, in order to further capture the context relationship in the target log sequence, especially the sequence information between the log templates, the log template matrix can be input into two layers of T-Encoders for further encoding, and the T-Encoders model the sequence data through the MHSA module and the MLP module.

[0079] Before entering the T-Encoders, a position information matrix is generated using the sine-cosine position encoding method, and is added to the log template matrix The calculation formula of the position encoding can refer to formula (1) and formula (2).

[0080] (1)

[0081] (2)

[0082] In formula (1) and formula (2), represents the position index, represents the index of the dimension in the log template vector, represents the dimension of the vector.

[0083] Next, the calculation process of the MHSA is described in detail. Let the input of the T-Encoder of the layer be , the number of attention heads be H , and it is usually required that can be divided by HEach attention head has latitude layers query matrix key matrix and value matrix The calculation process of the query matrix

[0084] , the key matrix and the value matrix is as shown in equations (3) to (5).

[0085] (3)

[0086] (4)

[0087] (5)

[0088] l In equations (3) to (5), respectively represent the learnable parameter matrices for generating query, key and value representations. First, the input of the T-Encoder layer is linearly transformed to obtain the query matrix

[0089] , the key matrix and the value matrix The three learnable parameter matrices are randomly initialized and updated during the model training process. H Next, the query matrix , the key matrix and the value matrix are divided into sub-matrices according to the feature latitude, and for each attention head, the query matrix, the key matrix and the value matrix of the current attention head

[0090] are ,

[0091] , respectively.

[0092] Further, the calculation process of equations (6) to (8) is performed for each attention head to capture the context relationship of the target log sequence. H (6)

[0093] The outputs of attention heads are concatenated in the column direction and linearly projected to obtain the input

[0094] of the T-Encoder layer.

[0095] In formula (6) to formula (8), denotes the output of the current attention head , denotes the output of the current attention head H concatenated in the column direction, denotes a projection matrix. Here, only the key multi-head attention layer is described in detail, and the multi-layer perception, residual connection and layer normalization are not described. After encoding by the double-layer T-Encoder, the log template feature matrix containing the context information of the target log sequence is obtained .

[0096] Step 104, input the log template feature matrix, the target log sequence and the log parameter matrix into the classification task sub-model in the log anomaly detection model, perform overall anomaly detection on the target log sequence based on the log template feature matrix, at the same time, perform local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix, the local anomaly detection adopts a determination strategy of a dynamic threshold function based on sequence length combined with a voting mechanism; determine the final anomaly detection result of the target log sequence based on the overall anomaly detection result and the local anomaly detection result.

[0097] Specifically, through the classification task sub-model in the log anomaly detection model, the target log sequence can be subjected to sequence-level anomaly detection and fine-grained anomaly detection of individual log instances, that is, two different level detection strategies designed for the target log sequence are sequence detection task and single log detection task.

[0098] For the sequence detection task, sequence-level anomaly detection can be achieved by aggregating the log template feature matrix of the entire target log sequence; for the single log detection task, fine-grained anomaly detection of single template log can be achieved by considering the log template feature and the log parameter feature of the single target log at the same time.

[0099] And the final anomaly detection result of the target log sequence needs to be determined by the two detection tasks, for example, if the determination results of the two detection tasks are both normal, it can be determined that the target log sequence is normal, and if the determination result of at least one of the two detection tasks is abnormal, it can be determined that the target log sequence is abnormal.

[0100] The log anomaly detection method provided by the embodiment of the application, by pre-training the SBERT sub-model, vectorizes the initial token sequence of each target log in the target log sequence, by the feature extraction sub-model, determines the log parameter matrix of the target log sequence and the log template feature matrix containing the sequence context information in the target log sequence based on the initial token vector matrix corresponding to each initial token sequence, and by the classification task sub-model, performs sequence-level anomaly detection and single log anomaly detection on the target log sequence in parallel, and determines the final anomaly detection result of the target log sequence based on the anomaly detection results of the two different levels. In this way, not only can the pre-trained SBERT model avoid the assumption of closed and stable log mode for the model, but also by designing detection tasks of different granularities, parallel anomaly detection is performed from two granularities of log sequences and single logs, respectively. The sequence detection task only uses template information to avoid noise introduced by dynamic parameters; the single log detection task combines template and parameter information to retain the complete features of log information. The two detection tasks share the feature extraction sub-model of the model and mutually serve as regular terms for the synergistic effect of the double tasks, so that the model can more comprehensively identify anomalies, improve the generalization ability and detection precision of the model, thereby ensuring the anomaly detection precision of complex and variable log sequences.

[0101] Based on the above Figure 1 In an example embodiment, the method shown in the step 101 determines the initial token sequence of each target log in the target log sequence. In this embodiment, the specific process can be implemented by the steps 201 and 202 shown in the following. Figure 2

[0102] The step 201 pre-processes the original log sequence collected from the target application software to obtain the target log sequence; wherein the pre-processing includes deleting empty logs and logs containing missing fields in the original log sequence, replacing spaces of special characters in the logs and merging adjacent continuous spaces, and standardizing the content format in the logs.

[0103] The step 202 tokenizes each target log in the target log sequence to obtain the initial token sequence of each target log.

[0104] Each collected original log is usually semi-structured text and can include but is not limited to timestamp, event type, log level, message content, etc.

[0105] Specifically, the original logs are collected from the target application software. The collection method can be real-time collection or periodic collection, or other pre-prepared collection methods can be used for collection; here is not limited.

[0106] ​In order to ensure the quality of log data and provide effective input for subsequent analysis, the raw log sequence containing a large number of raw logs collected can be preprocessed, and the preprocessing strategy can include but is not limited to: deleting empty logs and logs containing missing fields, writing a regular expression to uniformly replace special characters with spaces and merge adjacent continuous spaces, and converting all log contents to lowercase to avoid the problem of inconsistent sizes; so as to obtain the target log sequence.

[0107] For example, when a raw log is "DHCPOFFER on 10.100.4.251 to 00:11:43:e3:ba:c3 via eth1", the target log obtained after preprocessing is "dhcpoffer on 10.100.4.251 to 00:11:43:e3:ba:c3 via eth1". In this way, the raw log sequence After preprocessing, the target log sequence represents the length of the raw log sequence, represents the length of the target log sequence.

[0108] For each target log in the target log sequence, a log parser can be used to further parse each target log; the log parser can separate each target log into a log template and a log parameter, wherein the log template refers to the static part in the corresponding template log and is composed of fixed keywords, representing the structured part of the corresponding target log; the log parameter refers to the dynamic part in the corresponding template log, i.e. the content marked by the mask label, such as timestamp, IP address, etc.; the present application uses an open source tool (such as Drain) as a log parser, but is not limited to a specific tool, and a suitable log parser can be selected according to actual needs.

[0109] For example, the log parser , the target log sequence outputs a log template set and a log parameter sequence after processing by the log parser, and the process refers to formula (9).

[0110] (9)

[0111] In formula (9), represents the log template set, represents the number of log templates, which is generally represents the length of the raw log sequence, represents the log template corresponding to the th target log in the log template set; represents a sequence of log parameters, represents a log parameter corresponding to the target log in the target log sequence. For any target log in the target log sequence , it must match one of the log templates in the log template set , which can be represented as , where and are not necessarily the same, that is, multiple template logs can correspond to the same log template, but their parameter parts are different.

[0112] For example, the target log =“ dhcpoffer on 10.100.4.251 to 00:11:43:e3:ba:c3 via eth1 ” is separated into =“ dhcpoffer on <*> to <*> via <*> ”, = (“ 10.100.4.251 ”, “ 00:11:43: e3:ba:c3 ”, “ eth1 ”).

[0113] When the original log sequence contains 4 original logs, the 4 target logs obtained after preprocessing and the 4 log parameters obtained after parsing can be referred to as shown in Figure 3 .

[0114] Based on the method shown in Figure 1 , in an example embodiment, the step of determining the log parameter matrix and the log template matrix of the target log sequence from the plurality of initial token vector matrices in step 103 can be implemented by steps 301 and 302 shown in the embodiment. Figure 4 .

[0115] Step 301, decompose each initial token vector matrix into an initial token template matrix and an initial token parameter matrix.

[0116] Step 302, respectively aggregate a group of initial token template vectors contained in each initial token template matrix and a group of initial token parameter vectors contained in each initial token parameter matrix to obtain a log parameter matrix and a log template matrix.

[0117] Specifically, for any target log in the target log sequence , its binary mask vector can be generated based on the log parsing result of the target log and the word segmentation result output by the pre-trained SBERT submodel, where represents hthe k tokens belong to the log template, h tokens belong to the log parameter, k h tokens represent the target log . The initial token vector matrix of the target log is divided into a vector corresponding to the log template and a vector corresponding to the log parameter based on a binary mask vector , where and are index sets of the log template tokens and the log parameter tokens, respectively. .

[0118] The initial token vector matrix of the target log can be decomposed into an initial token template matrix and an initial token parameter matrix ; wherein, Here, the vectors corresponding to the tokens representing the template and the parameter can be extracted from the initial token vector matrix of the target log , that is, the initial token template matrix and the initial token parameter matrix of the target log are extracted, by a feature filter. Subsequently, the initial token template matrix and the initial token parameter matrix

[0119] of the target log are processed, that is, the initial token template matrix of the target log is aggregated into a template vector and the initial token parameter matrix is aggregated into a parameter vector by a mean pooling layer, representing the template information and the parameter information of the target log , respectively, providing more detailed feature representation for the subsequent network. The calculation formula of the mean pooling layer is referred to formula (10) and formula (11). (10)

[0120] (11)

[0121]

[0122] ​​​​​Similarly, each target log in the target log sequence is processed by the above-mentioned processing mode to obtain the template vector and the parameter vector corresponding to each target log, and then a log template matrix at a sequence level is obtained and a log parameter matrix .

[0123] Based on the above Figure 1 In an example embodiment, the step 104 of performing overall anomaly detection on the target log sequence based on the log template feature matrix, the specific process of which in this embodiment can be implemented by steps 401 to 404 shown in the following. Figure 5

[0124] Step 401, based on the query vector and the log template feature matrix, determining an attention weight vector of the target log sequence.

[0125] Step 402, performing Sofmax normalization processing on the attention weight vector, and aggregating all column vectors in the log template feature matrix based on the normalized attention weight vector to obtain a sequence feature column vector of the target log sequence, the sequence feature column vector being used to represent the overall sequence information of the target log sequence.

[0126] Step 403, performing feature extraction and merging on the sequence feature column vector according to the fully connected layer parameters to obtain a sequence anomaly real number representing the overall anomaly of the target log sequence.

[0127] Step 404, mapping the sequence anomaly real number to a preset numerical range, and the mapping result is a sequence anomaly probability of the target log sequence and belongs to the overall anomaly detection result.

[0128] Specifically, the classification task sub-model provided by the present application is a model obtained by jointly training a sequence anomaly detection task and a single log detection task. Specifically, the sequence anomaly detection task focuses on identifying the abnormal pattern of the target log sequence from a global perspective, and the single log detection task focuses on the local abnormal behavior of a single target log. Since the two tasks have high correlation in anomaly detection, the present application adopts a hard parameter sharing mechanism, that is, the parameters are shared in the shallow feature extraction part of the model, and the deep part is processed by independent task heads for the two tasks, that is, the model is forced to infer the general feature representation applicable to both sequence anomaly detection and single log anomaly detection. The two detection tasks serve as each other's regularization term, improving the robustness of the feature space to changes in log semantic patterns and effectively reducing the risk of overfitting of the model, thereby realizing collaborative optimization and resource sharing between tasks.

[0129] ​The detection process of the sequence detection task is described below; considering that different log instances in a log sequence may have different contributions to the model decision, the importance of these log instances needs to be distinguished. Specifically, the query vector and the log template feature matrix with context information are used to obtain the attention weight vector of the target log sequence The attention weight vector is normalized by the Softmax layer to obtain the normalized attention weight vector Then, the normalized attention weight vector is used to aggregate all column vectors in the log template feature matrix to generate a single column vector representing the information of the entire target log sequence, that is, the sequence feature column vector of the target log sequence The calculation process can refer to equation (12).

[0130] (12)

[0131] The sequence feature column vector is sent to two fully connected layers, using the Rectified Linear Unit (RELU) as the activation function, and then using the Sigmoid function to map the output to 0-1, outputting the overall anomaly probability distribution of the sequence, that is, the sequence anomaly probability of the target log sequence, and the entire process can refer to equations (13) to (16).

[0132] (13)

[0133] (14)

[0134] (15)

[0135] (16)

[0136] In equations (13) to (16), represents feature extraction and merging of the sequence feature column vector according to the fully connected layer parameters, and represent two weight matrices of the fully connected layer, and represent two bias vectors of the fully connected layer, , , and are all fully connected layer parameters. a sequence anomaly probability representing a target log sequence; a mapping function representing a real number range, a preset numerical value range is .

[0137] It should be noted that, for the model training process, a randomly initialized query vector is introduced in the attention pooling module, and the query vector is projected through a parameter learnable projection matrix, so that the query vector has a certain generalization and is considered as a query representation of which log instance is more important.

[0138] Based on the above Figure 1 The method shown in the figure, in an example embodiment, the step 104 based on the log template feature matrix and the log parameter matrix is locally abnormal for the target log sequence, and the specific process in this embodiment can be realized by Figure 6 The steps 501 to 503 shown in the figure.

[0139] Step 501, the log template feature vector corresponding to the target log in the log template feature matrix is spliced with the log parameter vector corresponding to the target log in the log parameter matrix to obtain the log feature splicing vector of the target log.

[0140] Step 502, according to the full connection layer parameter, the log feature splicing vector is extracted and combined to obtain the log abnormal real number representing the abnormal situation of the target log.

[0141] Step 503, mapping the log abnormal real number in a preset numerical value range, the mapping result is the log abnormal probability of the target log and belongs to the local abnormal detection result.

[0142] Specifically, since the classification task sub-model provided by the application is obtained by jointly training the sequence anomaly detection task and the single log detection task. Therefore, in the case of the detection process of the aforementioned sequence detection task, the detection process of the single log detection task is described here.

[0143] In the case that the log template feature matrix with context information , and the log parameter matrix have been obtained, in order to combine the parameter and template information, the log parameter vector and the log template feature vector corresponding to each target log can be spliced to obtain the log feature splicing vector v of the target log, the process of which can be referred to formula (17).

[0144] (17)

[0145] In equation (17), This indicates a splicing operation. Indicates the dimension of the vector.

[0146] Next, the log features of the target log are concatenated into a vector. The output is fed into two fully connected layers in sequence, using ReLU as the activation function, and then using the Sigmoid function to map the output to between 0 and 1, outputting the abnormal probability distribution of a single target log, that is, the log abnormal probability of the target log. The specific process can be referred to in equation (18).

[0147] (18)

[0148] In equation (18), Mapping functions representing the range of real numbers, Indicates the first [number]th [item] in the target log sequence i The probability of log anomalies in each target log. This represents concatenating a vector from log features based on the parameters of the fully connected layer. Perform feature extraction and merging; preset numerical range is .

[0149] Repeat the above process to traverse all target logs in the target log sequence, thereby obtaining the log anomaly probability for each target log.

[0150] Based on the above Figure 1 In one example embodiment of the method shown, step 104 determines the final anomaly detection result of the target log sequence based on the overall anomaly detection result and the local anomaly detection result. The specific process of this step in this embodiment can be achieved through… Figure 7 Steps 601 to 604 shown are implemented.

[0151] Step 601: When the local anomaly detection results include the log anomaly probability of each target log in the target log sequence, the abnormal logs in the target log sequence are determined based on the relationship between the anomaly probability of each log and the preset log anomaly probability threshold; wherein, the preset log anomaly probability threshold is determined based on a dynamic threshold function.

[0152] Step 602: Based on the relationship between the proportion of abnormal logs in the target log sequence and the preset proportion threshold, determine the first anomaly detection result of the target log sequence; wherein, the process of determining the first anomaly detection result is determined by a voting mechanism.

[0153] Step 603: Given that the overall anomaly detection result includes the sequence anomaly probability, determine the second anomaly detection result of the target log sequence based on the relationship between the sequence anomaly probability and the preset sequence anomaly probability threshold.

[0154] Step 604: Perform a logical OR operation on the first anomaly detection result and the second anomaly detection result to obtain the final anomaly detection result.

[0155] Specifically, during the inference phase, the classification task sub-model provides the final detection result by combining the anomaly probabilities output from the two detection tasks. An indicator function is defined. As shown in equation (19).

[0156] (19)

[0157] For sequence detection tasks, the probability of sequence anomalies in the target log sequence is: The anomaly determination of the target log sequence can be represented as: , This indicates the first anomaly detection result of the target log sequence, when When this occurs, it indicates that the task has determined the target log sequence to be abnormal.

[0158] For a single log detection task, considering that as the log sequence length increases, if a fixed threshold is used for anomaly detection of a single log entry, the probability of false positives will accumulate and increase, leading to a sharp increase in the overall false positive rate, affecting system stability and usability. For example, assuming the model's false positive rate for a single log entry is only 1%, if a fixed threshold of 0.5 is used as the judgment standard, and any abnormal log entry in the sequence is considered an anomaly for the entire sequence, then the overall false positive rate will increase significantly with the increase in sequence length. Specifically, when the sequence length is 10, the overall false positive rate is approximately... When the sequence length is 100, the overall false alarm rate can reach as high as approximately 63%. This phenomenon leads to a sharp increase in the false alarm rate when processing long log sequences, severely affecting the system's stability and practical application performance. To alleviate this problem, a dynamic threshold function is introduced, specifically a probability threshold function. The anomaly detection threshold for a single target log can be based on this probability threshold function. Adjustment, probability threshold function The definition is shown in equation (20).

[0159] (20)

[0160] In equation (20), This indicates the maximum increase in the control threshold. This indicates the rate at which the control threshold rises.

[0161] No. i The anomaly determination of a target log can be represented as follows: , Indicates when the independent variable Values The corresponding preset log anomaly probability threshold, This indicates the number of target logs in the target log sequence. When When, it indicates the first time in the task judgment target log sequence. i The target logs are all abnormal logs. A voting mechanism is then introduced, which determines whether an abnormal log is detected when the proportion of abnormal logs in the target log sequence exceeds a preset threshold. Only when the target log sequence is identified as abnormal can the process be referred to in equation (21).

[0162] (twenty one)

[0163] In equation (21), This indicates the number of target logs in the target log sequence. This represents the probability of a second sequence anomaly in the target log sequence. This indicates the first anomaly detection result of the target log sequence; when When this occurs, it indicates that the task has determined the target log sequence to be abnormal.

[0164] The final anomaly determination of the target log sequence is made by a combination of two detection tasks, that is, ,in Represents a logical OR operation. This represents the final anomaly detection result for the target log sequence; when When this occurs, it indicates that the model ultimately determines the target log sequence to be abnormal.

[0165] Understandably, from a software iteration perspective, changes in log patterns are usually smooth, meaning the differences in log patterns between versions are not significant. Furthermore, different software systems may exhibit similarities in their descriptions of similar problems. Therefore, the model should possess the ability to retain knowledge and continuously learn. Current technologies do not adequately address this issue, typically employing a full parameter update approach. This means that each software version requires retraining the model and updating all parameters, while simultaneously saving all parameters of the corresponding model for each version for future use. This approach not only consumes substantial computing power but also leads to increased storage space, raising maintenance costs and limiting its scalability and practicality in real-world applications.

[0166] To solve this problem, the application can introduce the idea of efficient parameter update for frequent updates of log systems and the needs of new application scenarios, aiming to quickly update the model in the new version or different log format scenarios using the knowledge in the existing model, rather than starting from scratch with full parameter training. There are usually two traditional methods in the prior art: one is to fine-tune the full parameters, which can fully adapt to new data but has high computational cost; the other is to freeze the parameters of the feature extraction network and only update the task-related module, but the performance may degrade due to the inability to update the key feature extraction network parameters.

[0167] The application realizes efficient parameter update based on the Adapter module, that is, two different Adapter modules (preset MLP-Adapter module and preset ATT-Adapter module) are inserted into each T-Encoder of the pre-trained SBERT submodel, and the insertion position and structure are as shown in Figure 8 The model only updates the parameters of the Adapter module and the task head when updating, and the other parameters of the network remain frozen. The task head part includes an attention pooling module and two classifiers corresponding to each detection task. In this way, when the log anomaly detection model needs to be retrained to adapt to a new version or a new scenario, all the parameters in the backbone network are frozen, and only the parameters of the inserted Adapter module and the two task heads are updated. This design significantly reduces the computational cost and storage cost of model updating, ensures that the model can retain and utilize existing knowledge in new tasks, and at the same time realizes plug-and-play of multiple versions through dynamic loading of different Adapter modules, reducing the storage cost of the model.

[0168] Taking any one log in the sample log sequence as an example, in the first training, the log will be directly input into the SBERT submodel to obtain its initial token vector matrix E In model updating, each T-Encoder in SBERT will introduce the above two Adapter modules, and the input log will be encoded into a new initial token vector matrix E , and the same training and inference strategy is adopted subsequently.

[0169] For each inserted preset MLP-Adapter module, the classic form of Adapter is used, and the calculation method is as shown in equation (22).

[0170] (22)

[0171] In equation (22), represents a dimension reduction matrix and is used to reduce the dimension of the input vector, denotes the dimension lifting matrix and is used to restore the original dimension of the vector. Each preset MLP-Adapter module and the corresponding multi-layer perceptron (MLP) module are inserted in parallel, and the preset MLP-Adapter module is denoted as is the output vector after the MHSA module, the preset ATT-Adapter, and the normalization module, and the calculation process is shown in equation (23).

[0172] (23)

[0173] Parallel insertion provides two feature transformation paths for the model, one is the original multi-layer perceptron path, and the other is the Adapter path specially adjusted for new features. Combining the two pieces of information can obtain richer and more robust feature representations.

[0174] Considering that the MLP-Adapter may not be sufficient in capturing global dependencies and contextual semantics, and in order to maintain higher consistency with the original model structure, the low-rank decomposition strategy is adopted, and a preset ATT-Adapter module based on the low-rank strategy and attention mechanism is proposed, and the calculation process is shown in equations (3) to (8). The key is that the parameter matrix for mapping QKV and the final dimension reduction in the preset ATT-Adapter module uses a low-rank matrix with a dimension of instead of , where Compared with the original self-attention module, the parameter amount of each mapping matrix is greatly reduced from to . Taking the output dimension of the SBERT submodel as an example, when r = 32 is selected, the parameter amount of each mapping matrix is greatly reduced from 768x768 to 768x32, and the parameter amount is reduced by 95.83%.

[0175] Each preset ATT-Adapter module is inserted in series after the corresponding multi-head self-attention mechanism (MSHA) module, and is denoted as is the output of the previous T-Encoder, and the calculation process is shown in equation (24).

[0176] (24)

[0177] This design not only maintains the advantages of self-attention mechanism in capturing global semantics and dependencies, but also only updates a small number of parameters during fine-tuning, thereby balancing the update efficiency and performance. The MHSA module is responsible for capturing global dependencies and contextual semantics, and the preset ATT-Adapter inserted in series can further finely adjust these global features. This insertion method ensures that the ATT-Adapter only supplements and fine-tunes the existing global features.

[0178] In this invention, the preset MLP-Adapter (r=32) and preset ATT-Adapter (r=32) inserted in each T-Encoder layer require updates of approximately 0.58 M and 1.18 M parameters, respectively. The feature extraction layer has approximately 124.55 M parameters, and the task header has approximately 2.36 M parameters. Compared to full fine-tuning, the model update method proposed in this invention updates only 3.25% of the parameters required for full fine-tuning (0.58 + 1.18 + 2.36) / (110 + 14.55 + 2.36) = 3.25%.

[0179] The model is first pre-trained in an initial or stable version of the software, employing a full parameter update strategy and saving all model parameters. Subsequently, when updating the model (e.g., during a software update), the feature extraction layer of the initial model is frozen, and only the parameters of the inserted Adapter module and task header are updated. Because only the parameters of the Adapter module and task header are updated, subsequent versions only need to save these incremental parameters, while the core parameters of the pre-trained model remain unchanged. This modular design allows the Adapter to be replaced independently as a plugin, enabling rapid switching between versions. In this way, the model significantly reduces the number of parameters updated and can smoothly migrate between different software versions.

[0180] Reference Figure 9 The diagram shown illustrates the structure of the multi-task learning framework. Figure 9 In the model, the feature extraction layer includes a pre-trained SBERT sub-model and a feature extraction sub-model. The pre-trained SBERT sub-model is... Figure 9 The Sentence-BERT sub-model for classification tasks is specifically as follows: Figure 9 It contains two classifier modules and an attention pooling module.

[0181] The following is combined with Figure 9 The training process of the log anomaly detection model is described in detail below:

[0182] First, a large amount of raw log data is collected from the target software system or other application software and preprocessed. All the preprocessed logs are used as sample logs in the sample log sequence. The sample log sequence can be used for model training or for the retraining process of the model where parameters are efficiently updated. The preprocessing process involved can be referred to the aforementioned embodiments; it will not be repeated here.

[0183] Secondly, on the basis of pre-processing, the log parser is used to further parse each sample log in the sample log sequence. The log parser separates each sample log into a log template and a log parameter, wherein the log template refers to the static part in the corresponding sample log, which is composed of fixed keywords; the log parameter refers to the dynamic part in the corresponding log, i.e. the content marked by the mask label; wherein the parsing process involved can refer to the foregoing embodiments; hereinafter, no longer be described. Then, the sample log sequence is cut to construct the input data acceptable to the model; the present application adopts the cutting strategy of fixed number of logs to cut the sample log sequence into a plurality of continuous and non-overlapping log subsequences; that is, the cutting window size is set to n , the sample log sequence is divided into log subsequences, denoted as , wherein the i th log subsequence , the log subsequence is a sample for model input; the sample log sequence is cut into a plurality of log subsequences containing a fixed number of logs, which are used as the input of the subsequent multi-task learning framework.

[0184] Then, the model is first trained on the annotated dataset. Tokenization is performed on each sample log in the sample log sequence, and the token-level vector representation of each sample log is obtained by inputting it into the pre-trained SBERT sub-model and taking the output of the aggregated last hidden layer. Subsequently, the vectors corresponding to the tokens representing the template and the parameter are extracted respectively by a feature filter (i.e. Filter in Figure 9 ). The token-level vectors of the template and the parameter are respectively subjected to mean pooling operation to aggregate the tokens of each log instance, and the parameter vector and the template vector of each sample log are respectively generated. To introduce sequence position information, the template vector is positionally encoded using the sine-cosine position encoding, and the template vector corresponding to each sample log in the corresponding sample log sequence is respectively input into two layers of T-Encoder to capture the context dependency within the sample log sequence. Through the stacking of multiple layers of T-Encoder, the model can effectively capture the long-distance dependency in the sequence and enhance the expression ability of the features.

[0185] Taking the i th log subsequence s i , the sample log parameter matrix of the i th log subsequence and the iThe sample log template feature matrix of the sequence context information in the log sub-sequence. In this way, the log sequence template and the parameter are extracted respectively, and the context relationship of the log sequence template is modeled, providing high-quality input features for subsequent multi-task learning. The model will further combine the multi-task learning framework to jointly optimize the loss for different types of anomaly detection tasks, thereby improving the accuracy and robustness of detection.

[0186] Finally, the method designs two different levels of detection strategies for logs: 1. Sequence detection task: globally aggregate the template information of the entire log sequence to realize sequence-level anomaly detection; 2. Single log detection task: consider the template and parameter information of a single log instance to realize fine-grained anomaly detection of a single log instance. Through hard sharing of the bottom feature extraction module, the model is forced to learn a general feature representation suitable for both sequence anomaly detection and single instance anomaly detection. The two tasks serve as regularizers for each other, improving the robustness of the feature space to changes in log semantic patterns and effectively reducing the risk of model overfitting. During training, both tasks use the cross-entropy loss function, and the model is trained end-to-end through joint loss. The processing flow of the two task heads is described in detail below.

[0187] Sequence detection task: at this time, the sample log parameter matrix of the first i log sub-sequence and the sample log template feature matrix containing the sequence context information in the first i log sub-sequence have been obtained , according to the determination process of the target log sequence, the log anomaly probability of the first i target log in the target log sequence and the sequence anomaly probability of the target log sequence , the log anomaly probability of the first i sample log in the first j log sub-sequence and the sequence anomaly probability of the first i log sub-sequence can be obtained.

[0188] For the single log detection task, since a log sub-sequence contains n sample logs, the detection task will ultimately output n log prediction probabilities , and the classification heads of the two detection tasks have their own independent parameters. In the training phase, the cross-entropy loss function is used as the loss function for the two detection tasks, and the loss function of the sequence detection task refers to formula (25).

[0189] (25)

[0190] Loss function for a single log inspection task Refer to formula (26).

[0191] (26)

[0192] In equations (25) and (26), Represents the sample log sequence Size, Indicates the first... i Labels of log subsequences Indicates the first... i In the log subsequence, the first j The labels for each sample log are 0 for normal and 1 for abnormal. Indicates the first i In the log subsequence, the first j Log anomaly probability of a sample log. Indicates the first i The probability of sequence anomalies in a log subsequence.

[0193] In multi-task learning, the loss functions for different detection tasks may have different scales and difficulties. Directly using a fixed weight combination can easily lead to one task dominating the overall loss during training. Furthermore, determining the optimal weights requires extensive experimentation. To mitigate this issue, this invention introduces uncertainty modeling to automatically learn the noise level of each task and dynamically adjust the weights of each task's loss term in the joint loss. That is, it introduces two positive, learnable scalars. and , representing the inherent uncertainty of each detection task, and the joint loss function of the two detection tasks. Refer to formula (27).

[0194] (27)

[0195] In equation (27), The loss function for sequence detection tasks The weight, The loss function for a single log inspection task The weights; As a regular expression term, to prevent and The weights tend towards positive infinity, avoiding a tendency for the weights to approach zero. and The initial values ​​of all values ​​are 1, representing the initial assumption that the noise of the two detection tasks is equal.

[0196] The training process adopts a gradient update mechanism based on a small batch of samples. For each iteration, a batch is sampled from the training set log subsequence, and the batch is input into the log sequence anomaly detection task and the log instance anomaly detection task. The corresponding joint loss function value is calculated, and the model parameters are updated based on the loss value using the Adam optimizer.

[0197] After each iteration, the performance indicator (such as F1-Score) of the current model on the validation set is calculated, and the early stopping strategy is used. When the performance indicator does not significantly improve for several rounds, it is determined that the model has converged, the training process is terminated, and the model at this time is saved as the final log anomaly detection model. Otherwise, the next batch is sampled from the training set, and the training continues until the training termination condition is met.

[0198] It should be noted that the training phase and the inference phase have consistency in the processing flow, and both perform log instance-level and log sequence-level anomaly detection on the input log sequence and output the corresponding anomaly probability. The main difference between the two is that the training phase needs to calculate the joint loss based on the anomaly probability output by the two sub-tasks and the sample label to drive the model parameter optimization and evaluate the convergence. The inference phase no longer performs parameter update, but directly determines the final anomaly judgment output based on the detection results of the two sub-tasks and the inference strategy. Therefore, compared with the prior art, the present application has the following technical effects:

[0199] (1) Multi-granularity structured semantic modeling: Based on the token-level representation ability of the pre-trained language model SBERT, the complete log text is input into the model, and the template and parameter are divided according to the token position, and the vector representation is formed by aggregation. This method not only enhances the modeling ability of the potential dependency relationship between the template and the parameter, but also improves the fault tolerance and the overall robustness of the model under the condition of parsing errors.

[0200] (2) Task-collaborative multi-view anomaly detection mechanism: The present application constructs log sequence detection and single log detection tasks, which respectively face the global behavior pattern of the log sequence and the local semantics of the single log. The former only models the context structure based on the template vector, and the latter uses the parameter and template vector to improve the fine-grained recognition ability. The double tasks share the feature extraction module and are jointly trained, which has the characteristics of structural orthogonality and semantic complementarity, significantly improving the comprehensiveness, accuracy of anomaly detection and the generalization ability of the model.

[0201] (3) Dual-task joint discrimination mechanism: In view of the problem that false positives are easy to accumulate in long log sequences, the application proposes a dynamic threshold adjustment mechanism based on sequence length, and combines the abnormal log proportion for joint determination, effectively alleviating false positive propagation and improving the stability and applicability of the model in the actual deployment environment.

[0202] (4) Efficient model updating: By introducing two lightweight modules, MLP-Adapter in parallel and ATT-Adapter in series, into the main model, keeping the main model parameters frozen, and only updating the Adapter and task header, the rapid iteration and adaptation of the model can be completed. This structure has the advantages of pluggable and migratable, significantly reducing the parameter update overhead, and is suitable for online deployment and long-term maintenance requirements under frequent software version evolution.

[0203] The application provides a flexible, high-precision and robust log anomaly detection method by combining the multi-task learning paradigm and the efficient parameter migration mechanism, which is particularly suitable for complex scenarios with unstable log structure and frequent system evolution.

[0204] It should be noted that although the operations of the method of the application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all of the shown operations must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can change the order of execution. Additionally or alternatively, certain steps can be omitted, combined into one step, and / or divided into multiple steps.

[0205] In one embodiment, the application also provides a log anomaly detection device, as shown in Figure 10 The log anomaly detection device comprises a word sequence determination unit 1001, a log feature extraction unit 1002 and a log anomaly detection unit 1003.

[0206] The word sequence determination unit 1001 is configured to determine the initial word sequence of each target log in the target log sequence.

[0207] The log feature extraction unit 1002 is configured to input each initial word sequence into a pre-trained SBERT sub-model in the log anomaly detection model, vectorize each initial word sequence to obtain an initial word vector matrix corresponding to each initial word sequence; input each initial word vector matrix into a feature extraction sub-model in the log anomaly detection model, determine a log parameter matrix and a log template matrix of the target log sequence from the plurality of initial word vector matrices, and extract the context dependency relationship inside the target log sequence based on the log template matrix and the position information of the log template matrix to obtain a log template feature matrix containing the sequence context information in the target log sequence.

[0208] The log anomaly detection unit 1003 is configured to input the log template feature matrix, the target log sequence, and the log parameter matrix into a classification task submodel in the log anomaly detection model, perform overall anomaly detection on the target log sequence based on the log template feature matrix, and at the same time, perform local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix. The local anomaly detection adopts a determination strategy of a dynamic threshold function based on a sequence length combined with a voting mechanism. The final anomaly detection result of the target log sequence is determined based on the overall anomaly detection result and the local anomaly detection result.

[0209] In one embodiment, the log anomaly detection unit 1003 is specifically configured to determine an attention weight vector of the target log sequence based on the query vector and the log template feature matrix, perform Softmax normalization processing on the attention weight vector, aggregate all column vectors in the log template feature matrix based on the normalized attention weight vector, obtain a sequence feature column vector of the target log sequence, and use the sequence feature column vector to represent overall sequence information of the target log sequence. The sequence feature column vector is used to perform feature extraction and merging on the sequence feature column vector according to the full connection layer parameter, to obtain a sequence anomaly real number representing the overall anomaly situation of the target log sequence. The sequence anomaly real number is mapped to a preset numerical range, and the mapping result is a sequence anomaly probability of the target log sequence and belongs to the overall anomaly detection result.

[0210] In one embodiment, the log anomaly detection unit 1003 is specifically configured to perform splicing processing on a log template feature vector corresponding to the target log in the log template feature matrix and a log parameter vector corresponding to the target log in the log parameter matrix, to obtain a log feature splicing vector of the target log. The log feature splicing vector is used to perform feature extraction and merging on the log feature splicing vector according to the full connection layer parameter, to obtain a log anomaly real number representing the anomaly situation of the target log. The log anomaly real number is mapped to a preset numerical range, and the mapping result is a log anomaly probability of the target log and belongs to the local anomaly detection result.

[0211] In one embodiment, the log anomaly detection unit 1003 is specifically configured to, in a case where the local anomaly detection result comprises a log anomaly probability of each target log in the target log sequence, determine an abnormal log in the target log sequence based on a size relationship between each log anomaly probability and a preset log anomaly probability threshold; the preset log anomaly probability threshold is determined based on a dynamic threshold function; determine a first anomaly detection result of the target log sequence based on a size relationship between a proportion of the abnormal log in the target log sequence and a preset proportion threshold; the process of determining the first anomaly detection result is determined by a voting mechanism; in a case where the overall anomaly detection result comprises a sequence anomaly probability, determine a second anomaly detection result of the target log sequence based on a size relationship between the sequence anomaly probability and a preset sequence probability threshold; and perform a logical OR operation on the first anomaly detection result and the second anomaly detection result to obtain a final anomaly detection result.

[0212] In one embodiment, the log anomaly detection apparatus provided by the present application further comprises a parameter efficient updating unit, configured to, in a case where each Transformer encoder in the pre-trained SBERT sub-model comprises a multi-head self-attention mechanism module and a multi-layer perception module, and the classification task sub-model comprises an attention pooling module and a classifier module, in response to a software version updating instruction, insert each preset MLP-Adapter module into the corresponding Transformer encoder in parallel with the corresponding multi-layer perception module, and insert each preset ATT-Adapter module after the corresponding multi-head self-attention mechanism module and in series with the multi-head self-attention mechanism module, to obtain an updated SBERT model; replace the pre-trained SBERT model with the updated SBERT model to obtain an updated log anomaly detection model; and retrain the updated log anomaly detection model using a sample log sequence; the retraining comprises parameter updating training of module parameters of each preset MLP-Adapter module, each preset ATT-Adapter module, the attention pooling module and the classifier module; the parameter matrix in the preset ATT-Adapter module uses a low-rank matrix of a preset dimension; and the retrained log anomaly detection model is used for log anomaly detection of a new target log sequence.

[0213] In one embodiment, the log feature extraction unit 1002 is specifically configured to decompose each initial token vector matrix into an initial token template matrix and an initial token parameter matrix; and perform vector aggregation on a group of initial token template vectors contained in each initial token template matrix and a group of initial token parameter vectors contained in each initial token parameter matrix, respectively, to obtain a log parameter matrix and a log template matrix.

[0214] In an embodiment, the word sequence determining unit 1001 is specifically configured to preprocess the original log sequence collected from the target application software to obtain a target log sequence; wherein the preprocessing includes deleting empty logs and logs containing missing fields in the original log sequence, replacing spaces of special characters in the logs with spaces, merging adjacent continuous spaces, and standardizing the content format in the logs; each target log in the target log sequence is subjected to word segmentation to obtain an initial word sequence of each target log.

[0215] It should be understood that the units described in the log anomaly detection apparatus correspond to the respective steps in the method described above. Figure 1 The operations and features described above with respect to the method are also applicable to the log anomaly detection apparatus and the units included therein, and thus are not described herein again. The log anomaly detection apparatus can be pre- implemented in a browser or other security application of a computer device, or can be loaded into the browser or the security application thereof of the computer device through downloading or the like. The corresponding units in the log anomaly detection apparatus can cooperate with the units in the computer device to implement the solutions of the embodiments of the present application.

[0216] Reference is made below to Figure 11 which shows a structural schematic diagram of a computer system 1100 of a computer device or a server suitable for implementing the embodiments of the present application.

[0217] As shown in Figure 11 , the computer system 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1102 or loaded into a random access memory (RAM) 1103 from a storage portion 1108. In the RAM 1103, various programs and data required for the operation of the system 1100 are also stored. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0218] The following components are connected to the I / O interface 1105: an input part 1106 including a keyboard, a mouse, etc.; an output part 1107 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1108 including a hard disk, etc.; and a communication part 1109 including a network interface card such as a LAN card, a modem, etc. The communication part 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as necessary. A removable media 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1110 as necessary, so that a computer program read out therefrom is installed in the storage part 1108 as necessary.

[0219] In particular, in accordance with embodiments of the present application, the processes described above with reference to Figure 1 may be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program tangibly embodied on a machine readable medium, the computer program comprising program code for executing Figure 1 the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via the communication part 1109 and / or installed from the removable media 1111.

[0220] It should be noted that the computer-readable medium shown in the application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In this application, the computer-readable signal medium can include a data signal carried in a baseband or as a carrier wave in a carrier wave, which carries computer-readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0221] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the figures. For example, two blocks that are shown in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0222] The units or modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. The described units or modules can also be arranged in a processor. In some cases, the names of the units or modules do not constitute a limitation on the units or modules themselves.

[0223] As another aspect, the present application also provides a computer readable storage medium, which can be included in the computer device described in the above embodiments, or can exist separately and not be assembled into the computer device. The above computer readable storage medium stores one or more programs, when the programs are used by one or more processors to execute the methods described in the present application. For example, the following steps of the method shown can be executed. Figure 1 The steps of the method shown.

[0224] The embodiments of the present application provide a computer program product, which includes instructions that, when executed, cause the method described in the embodiments of the present application to be performed. For example, the following steps of the method shown can be executed. Figure 1 The steps of the method shown.

[0225] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0226] The above description is merely preferred embodiments of the present application and a description of the principles of the technology used. Those skilled in the art should understand that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or equivalent features without departing from the inventive concept. For example, the above features can be replaced with technical features with similar functions disclosed in the present application (but not limited to) to form technical solutions.

Claims

1. A log anomaly detection method, characterized by, The method comprises: determining an initial token sequence of each target log in a target log sequence; inputting each of the initial token sequences into a pre-trained SBERT submodel in a log anomaly detection model, vectorizing each of the initial token sequences to obtain an initial token vector matrix corresponding to each of the initial token sequences; inputting each of the initial token vector matrices into a feature extraction submodel in the log anomaly detection model, determining a log parameter matrix and a log template matrix of the target log sequence from the plurality of initial token vector matrices, and extracting a context dependency relationship within the target log sequence based on the log template matrix and position information of the log template matrix to obtain a log template feature matrix containing sequence context information in the target log sequence; inputting the log template feature matrix, the target log sequence, and the log parameter matrix into a classification task submodel in the log anomaly detection model, performing overall anomaly detection on the target log sequence based on the log template feature matrix, while at the same time, performing local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix, the local anomaly detection adopting a judgment strategy combining a sequence length-based dynamic threshold function with a voting mechanism; determining a final anomaly detection result of the target log sequence based on the overall anomaly detection result and the local anomaly detection result.

2. The method of claim 1, wherein, The overall anomaly detection on the target log sequence based on the log template feature matrix comprises: determining an attention weight vector of the target log sequence based on a query vector and the log template feature matrix; performing Softmax normalization processing on the attention weight vector, and aggregating all column vectors in the log template feature matrix based on the normalized attention weight vector to obtain a sequence feature column vector of the target log sequence, the sequence feature column vector being used to represent overall sequence information of the target log sequence; performing feature extraction and merging on the sequence feature column vector according to full connection layer parameters to obtain a sequence anomaly real number representing overall anomaly of the target log sequence; performing mapping of a preset numerical range on the sequence anomaly real number, and the mapping result is a sequence anomaly probability of the target log sequence and belongs to the overall anomaly detection result.

3. The method of claim 2, wherein, The local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix comprises: performing concatenation processing on a log template feature vector corresponding to the target log in the log template feature matrix and a log parameter vector corresponding to the target log in the log parameter matrix to obtain a log feature concatenation vector of the target log; performing feature extraction and merging on the log feature concatenation vector according to full connection layer parameters to obtain a log anomaly real number representing anomaly of the target log; performing mapping of a preset numerical range on the log anomaly real number, and the mapping result is a log anomaly probability of the target log and belongs to the local anomaly detection result.

4. The method of claim 3, wherein, The determining the final anomaly detection result of the target log sequence based on the global anomaly detection result and the local anomaly detection result comprises: In the case that the local anomaly detection result comprises a log anomaly probability of each target log in the target log sequence, determining an abnormal log in the target log sequence based on a size relationship between each log anomaly probability and a preset log anomaly probability threshold value, wherein the preset log anomaly probability threshold value is determined based on the dynamic threshold function; Determining a first anomaly detection result of the target log sequence based on a size relationship between a proportion of the abnormal log in the target log sequence and a preset proportion threshold value, wherein the process of determining the first anomaly detection result is determined by the voting mechanism; In the case that the global anomaly detection result comprises the sequence anomaly probability, determining a second anomaly detection result of the target log sequence based on a size relationship between the sequence anomaly probability and a preset sequence probability threshold value; Performing a logical OR operation on the first anomaly detection result and the second anomaly detection result to obtain the final anomaly detection result.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: In the case that each Transformer encoder in the pre-trained SBERT sub-model comprises a multi-head self-attention mechanism module and a multi-layer perception module, and the classification task sub-model comprises an attention pooling module and a classifier module, In response to a software version update instruction, inserting each preset MLP-Adapter module into the corresponding Transformer encoder in parallel with the corresponding multi-layer perception module, and inserting each preset ATT-Adapter module after the corresponding multi-head self-attention mechanism module and in series with the multi-head self-attention mechanism module to obtain an updated SBERT model; Replacing the pre-trained SBERT sub-model with the updated SBERT model to obtain an updated log anomaly detection model; Re-training the updated log anomaly detection model using a sample log sequence; the re-training comprises parameter update training of module parameters of each preset MLP-Adapter module, each preset ATT-Adapter module, the attention pooling module, and the classifier module, and the parameter matrix in the preset ATT-Adapter module uses a low-rank matrix of a preset dimension; the re-trained log anomaly detection model is used for log anomaly detection of a new target log sequence.

6. The method according to any one of claims 1 to 4, characterized in that, The determining the log parameter matrix and the log template matrix of the target log sequence from the plurality of initial token vector matrices comprises: Decomposing each initial token vector matrix into an initial token template matrix and an initial token parameter matrix; Respectively performing vector aggregation on a group of initial token template vectors contained in each initial token template matrix and a group of initial token parameter vectors contained in each initial token parameter matrix to obtain the log parameter matrix and the log template matrix.

7. The method according to any one of claims 1 to 4, characterized in that, The method comprises the following steps: The original log sequence collected from the target application software is preprocessed to obtain the target log sequence, wherein the preprocessing comprises deleting empty logs and logs containing missing fields in the original log sequence, replacing spaces of special characters in the logs with spaces, merging adjacent continuous spaces, and standardizing the content format in the logs; Each target log in the target log sequence is subjected to word segmentation to obtain the initial word sequence of each target log.

8. A log anomaly detection apparatus characterized by comprising: The device comprises: A word sequence determination unit configured to determine an initial word sequence of each target log in a target log sequence; A log feature extraction unit configured to input each initial word sequence into a pre-trained SBERT sub-model in a log anomaly detection model, vectorize each initial word sequence to obtain an initial word vector matrix corresponding to each initial word sequence, input each initial word vector matrix into a feature extraction sub-model in the log anomaly detection model, determine a log parameter matrix and a log template matrix of the target log sequence from a plurality of initial word vector matrices, and extract context dependency within the target log sequence based on the log template matrix and position information of the log template matrix to obtain a log template feature matrix containing sequence context information in the target log sequence; A log anomaly detection unit configured to input the log template feature matrix, the target log sequence, and the log parameter matrix into a classification task sub-model in the log anomaly detection model, perform overall anomaly detection on the target log sequence based on the log template feature matrix, at the same time, perform local anomaly detection on the target log sequence based on the log template feature matrix and the log parameter matrix, wherein the local anomaly detection adopts a judgment strategy combining a sequence length-based dynamic threshold function and a voting mechanism; and determine a final anomaly detection result of the target log sequence based on the overall anomaly detection result and the local anomaly detection result.

9. A computer device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the log anomaly detection method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the log anomaly detection method in any one of claims 1 to 7. The computer program is executed by the processor to implement the log anomaly detection method in any one of claims 1 to 7.

Images