Log anomaly detection method and device, equipment, medium and product

Abstract

The invention discloses a log anomaly detection method and device, equipment, a medium and a product, and relates to the field of log anomaly detection.The method comprises the steps that a to-be-detected log sequence is received; the to-be-detected log sequences are input into a log anomaly detection model to obtain the probability of abnormal logs in all the to-be-detected log sequences, the log anomaly detection model is obtained after an original anomaly detection model is trained, and a loss function in a weak supervision form is adopted when the original anomaly detection model is trained; determining that the corresponding to-be-detected log sequence is an abnormal log sequence when the abnormal log probability is greater than an abnormal threshold value; and outputting the abnormal log sequence. The accuracy of the abnormal log detection result can be improved.

Description

Technical Field

[0001] The present application relates to the field of log anomaly detection, and in particular to a log anomaly detection method, device, equipment, medium and product. Background Art

[0002] Under the wave of digital transformation, various information systems have become the core support for the operation of modern enterprises and organizations. When the system is running, a large amount of system logs will continue to be generated. These logs record in detail the operating status of the hardware and software in the system and related information of system problems, and can also monitor various events occurring in the system. From equipment failures and performance fluctuations at the hardware level, to program errors and module anomalies at the software level, to permission changes and network connection anomalies at the system level, the system log can record them completely. It is not only a "black box" for system operation, recording every move of the system in detail, but also the key to ensuring the stable operation of the information system, troubleshooting potential failures, and preventing security threats.

[0003] When a system fails, the log will also record abnormal information. This abnormal information is like a "clue" to the failure. Operation and maintenance personnel can obtain this key information by querying the log, so as to further track and locate the source of the failure.

[0004] As a key link in system log analysis, anomaly detection can timely identify abnormal conditions in system operation, thus providing strong support for system maintenance and security protection. In recent years, anomaly detection methods based on neural networks have been widely used in the field of system log analysis due to their powerful pattern recognition and data processing capabilities. By building a suitable neural network model, the system log data is deeply mined, and the difference characteristics between normal log patterns and abnormal log patterns are automatically learned, so as to accurately determine whether the new log is an abnormal log and output the abnormal log.

[0005] However, this solution is highly dependent on a large number of labeled data samples during model training. If there are incorrectly labeled samples in the samples, the performance of the trained log anomaly detection model will be reduced, making the detection results inaccurate. Summary of the invention

[0006] The purpose of this application is to provide a log anomaly detection method, device, equipment, medium and product, which can improve the accuracy of abnormal log detection results.

[0007] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a log anomaly detection method, comprising: Receive the log sequence to be detected; Inputting the to-be-detected log sequence into a log anomaly detection model to obtain the probability of abnormal logs in each of the to-be-detected log sequences, wherein the log anomaly detection model is obtained by training an original anomaly detection model, and a weakly supervised loss function is used when training the original anomaly detection model; Determine that the log sequence to be detected corresponding to when the abnormal log probability is greater than the abnormal threshold is an abnormal log sequence; Output the abnormal log sequence.

[0008] In a second aspect, the present application provides a log anomaly detection device, the log anomaly detection device comprising: A receiving module, used for receiving a log sequence to be detected; An acquisition module is used to input the log sequence to be detected into a log anomaly detection model to obtain the probability of abnormal logs in each of the log sequences to be detected, wherein the log anomaly detection model is obtained by training the original anomaly detection model, and a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function is used when training the original anomaly detection model; A detection module, used for determining that the corresponding log sequence to be detected is an abnormal log sequence when the abnormal log probability is greater than an abnormal threshold; An output module is used to output the abnormal log sequence.

[0009] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned log anomaly detection methods.

[0010] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the log anomaly detection methods described above.

[0011] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the log anomaly detection methods described above.

[0012] According to the specific embodiments provided in this application, this application discloses the following technical effects: The present application provides a log anomaly detection method, apparatus, device, medium and product, which receive a log sequence to be detected; input the log sequence to be detected into a log anomaly detection model to obtain the probability of abnormal logs in each of the log sequences to be detected, wherein the log anomaly detection model is obtained by training an original anomaly detection model, and a loss function in a weakly supervised form is used when training the original anomaly detection model; determine that the log sequence to be detected corresponding to when the probability of abnormal log is greater than an abnormal threshold is an abnormal log sequence; and output the abnormal log sequence.

[0013] In the present disclosure, since the log anomaly detection model adopts a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function during training, the contrastive learning loss function is a key component of the entire training process. It shortens the distance between similar samples and pushes the distance between dissimilar samples to enable the model to learn the intrinsic feature representation of the data. In the system log anomaly detection scenario, similar normal log samples will be close to each other in the feature space, while abnormal log samples will maintain a large distance from normal log samples, so that the model can clearly distinguish between normal and abnormal modes. The cross entropy loss function plays an important role in the supervised training link. When a small amount of labeled data is used for training, the cross entropy loss function can measure the difference between the model prediction results and the true labels, and make the prediction results as close to the true labels as possible by continuously adjusting the model parameters. This approach of combining the contrastive learning loss function with the cross entropy loss function gives full play to the advantages of the two loss functions. It not only uses contrastive learning to mine the intrinsic structure of the data, but also uses the cross entropy loss function to use a small amount of label information for precise guidance, so that the model can still learn efficiently under weak supervision conditions and accurately detect anomalies in system logs. That is, the present disclosure only relies on a small number of labels for supervised training, and most of it is based on self-supervised training without labels. On the one hand, it alleviates the impact of the scarcity of log labels on model training, and on the other hand, it reduces the training bias caused by noisy log labels, thereby improving the accuracy of the trained log anomaly detection model in detecting abnormal logs. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 This is a flow chart of a log anomaly detection method in an embodiment of the present application; Figure 2A flow chart of a method for extracting log semantic features from a training log sequence provided in one embodiment of the present application; Figure 3 A flow chart of a log sequence feature extraction method for a training log sequence provided in an embodiment of the present application; Figure 4 A schematic diagram of a comparative learning method provided in an embodiment of the present application; Figure 5 A schematic diagram of the functional modules of a log anomaly detection device provided in one embodiment of the present application; Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0018] Under the wave of digital transformation, various information systems have become the core support for the operation of modern enterprises and organizations. When the system is running, a large amount of system logs will continue to be generated. These logs record in detail the operating status of the hardware and software in the system and related information of system problems, and can also monitor various events occurring in the system. From equipment failures and performance fluctuations at the hardware level, to program errors and module anomalies at the software level, to permission changes and network connection anomalies at the system level, the system log can record them completely. It is not only a "black box" for system operation, recording every move of the system in detail, but also the key to ensuring the stable operation of the information system, troubleshooting potential failures, and preventing security threats.

[0019] Taking the e-commerce platform as an example, this is a very representative application scenario of log anomaly detection. The daily operation of the e-commerce platform involves multiple links such as product display, user browsing, order purchase, payment settlement, logistics distribution, etc. Each link will generate system logs, which include key data such as user ID, browsing product records, order time, payment amount, delivery address, and server response time. During major promotion activities, such as "Double 11" and "618", the e-commerce platform will usher in a peak in traffic. At this time, log anomaly detection can keenly capture the system operation status. If the server responds slowly due to excessive traffic, resulting in page loading timeout, or if the number of concurrent orders exceeds expectations, there are abnormal situations such as repeated order submission, payment failure but payment is displayed, etc., the log anomaly detection system will be quickly started. By analyzing the system logs in real time, the detection algorithm can promptly identify abnormal logs and issue alarms. Based on these abnormal logs, the technical team and operation personnel can quickly locate the problem whether it is due to excessive server load, database read and write bottlenecks, or vulnerabilities in the business logic code, and then take corresponding measures. For example, when an abnormal situation is discovered where a large number of users have failed to pay but the money has been deducted, the payment interface is immediately suspended to prevent more users from suffering losses, and technical personnel are organized to urgently investigate the problem, whether there is an abnormality in the connection between the payment interface and the bank system, or an error in the data transmission process, the problem is fixed in a timely manner and explained to the users, to ensure the normal trading order and user rights of the e-commerce platform.

[0020] When a system fails, the log will also record abnormal information. These abnormal information are like "clues" to the failure. Operation and maintenance personnel can obtain these key information by querying the log, so as to further track and locate the source of the failure. With the help of the timestamp, event details and related parameters recorded in the log, operation and maintenance personnel can sort out the context before and after the failure occurs, accurately determine whether the failure is caused by physical damage to the hardware, code vulnerabilities in the software, or unreasonable system configuration, and then take targeted measures to troubleshoot the failure and ensure that the system resumes normal operation as soon as possible. For example, when the system generates a log of "rts:kernel terminated for reason 1004", the operation and maintenance personnel can know that the system has a kernel interrupt exception based on the information "kernelterminated".

[0021] As a key link in system log analysis, anomaly detection can promptly identify abnormal conditions in system operation, thereby providing strong support for system maintenance and security protection.

[0022] However, as IT services develop, system components become larger in scale and the log data generated becomes more massive. For example, Google generates millions of new log entries every month, which means that TB-level log data is generated every day. It is inefficient and impractical to manually check for anomalies in log data. Secondly, a single developer or system administrator may not have sufficient knowledge of the entire system because many large enterprise systems often use commercial off-the-shelf components (such as third-party components). The field of artificial intelligence IT operations (AIOps) aims to enhance IT operations capabilities through advanced artificial intelligence algorithms to meet these challenges and further maintain the stability of enterprise services. With the development of machine learning, a log anomaly detection method based on machine learning has been proposed to automatically help operations personnel locate and detect anomalies in logs.

[0023] In the past few decades, many methods for log anomaly detection based on machine learning have been proposed. Some methods take quantitative log event counts as input and use traditional machine learning techniques to project event count vectors into vector space. Vectors that deviate from the majority or violate certain invariant relationships between event counts are classified as anomalies. Representative methods of traditional machine learning include logistic regression, single-class SVM, log clustering, and invariant mining. However, these methods based on traditional machine learning often use quantitative statistical data as features and cannot capture high-dimensional features such as rich semantic information in log text messages and sequential relationships between log event sequences. Therefore, their performance on different data sets is often unstable. Therefore, with the development of deep learning, many log anomaly detection methods based on deep neural networks have been proposed. By constructing appropriate neural network models, such as deep neural networks (DNNs), recurrent neural networks (RNNs) and their variants, long short-term memory networks (LSTMs), gated recurrent units (GRUs), etc., these methods can perform deep mining on system log data, better represent higher-dimensional features in logs, automatically learn the difference features between normal log patterns and abnormal log patterns, and then accurately determine whether new logs are abnormal logs and output abnormal logs. These deep learning-based log anomaly detection methods can therefore more accurately help operation and maintenance personnel locate and detect anomalies in logs.

[0024] However, we observe that existing deep learning-based log anomaly detection methods still have some limitations: (1) The superior performance of deep learning relies heavily on supervised training with sufficient labeled data. However, labeling large-scale data can be a difficult task that requires extensive expertise and is both laborious and time-consuming. As a result, obtaining labeled logs in real-world scenarios is scarce, which makes it difficult to scale to large datasets.

[0025] (2) There may be noisy labels (e.g., mislabeled samples) in the dataset, which has a great impact on the research of log anomaly detection based on deep learning. Noisy labels in the training set will reduce the performance of the log anomaly detection model trained with them, making the model less effective in practice. Second, noisy labels in the test / validation set may lead to misjudgment of the true performance of existing log anomaly detection methods.

[0026] (3) Since the system is always being updated and modified, developers may frequently change the source code, including log statements, which may cause changes in log-related data. Studies have found that approximately 20% to 45% of log statements change throughout their life cycle. Some changes to log output statements may be minor. For example, in order to express the log message more clearly, the developer changes the log output statement "Logger.info('Receiving block {block_id} src: / {src_ip} dest: / {dest_ip}')" to "Logger.info('Receiving block {block_id} source: / {src_ip} destination: / {dest_ip}')". However, the log parser will recognize it as a completely new log template by simply updating some tokens in the log text, thereby generating a new log. When this happens, anomaly detection models based only on log sequences will have an out-of-vocabulary problem, which may cause the model trained on the original log to misjudge unseen logs as unknown or minority classes (i.e., anomalies), resulting in false alarms.

[0027] In order to solve the above technical problems, the present disclosure proposes a log anomaly detection method, device, equipment, medium and product.

[0028] Figure 1 is a flow chart of a log anomaly detection method according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps S101-S104: In step S101, a log sequence to be detected is received.

[0029] In step S102, the log sequence to be detected is input into the log anomaly detection model to obtain the probability of abnormal logs in each log sequence to be detected, wherein the log anomaly detection model is obtained after training the original anomaly detection model, and a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function is used when training the original anomaly detection model.

[0030] In step S103, when it is determined that the abnormal log probability is greater than the abnormal threshold, the corresponding log sequence to be detected is an abnormal log sequence.

[0031] In step S104, an abnormal log sequence is output.

[0032] There are multiple log sequences to be detected, and then steps S102-S103 are performed to determine an abnormal log sequence from the multiple log sequences to be detected.

[0033] In the present disclosure, since the log anomaly detection model adopts a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function during training, the contrastive learning loss function is a key component of the entire training process. It shortens the distance between similar samples and pushes the distance between dissimilar samples to enable the model to learn the intrinsic feature representation of the data. In the system log anomaly detection scenario, similar normal log samples will be close to each other in the feature space, while abnormal log samples will maintain a large distance from normal log samples, so that the model can clearly distinguish between normal and abnormal modes. The cross entropy loss function plays an important role in the supervised training link. When a small amount of labeled data is used for training, the cross entropy loss function can measure the difference between the model prediction results and the true labels, and make the prediction results as close to the true labels as possible by continuously adjusting the model parameters. This approach of combining the contrastive learning loss function with the cross entropy loss function gives full play to the advantages of the two loss functions. It not only uses contrastive learning to mine the intrinsic structure of the data, but also uses the cross entropy loss function to use a small amount of label information for precise guidance, so that the model can still learn efficiently under weak supervision conditions and accurately detect anomalies in system logs. That is, the present disclosure only relies on a small number of labels for supervised training, and most of it is based on self-supervised training without labels. On the one hand, it alleviates the impact of the scarcity of log labels on model training, and on the other hand, it reduces the training bias caused by noisy log labels, thereby improving the accuracy of the trained log anomaly detection model in detecting abnormal logs.

[0034] In one embodiment, the original anomaly detection model includes a BERT (Bidirectional Encoder Representations from Transformers) model, a Transformer encoder, a feature fusion module, a regularization model, and a probability output model. The above method further includes the following sub-steps A1-A8: A1. Collect a training set, which includes a training log sequence and a training label corresponding to each training log sequence.

[0035] Collect training set , among which, Indicates training log sequences, M represents the number of training log sequences, Indicates The training labels corresponding to the training log sequences are , where 0 means normal, 1 means abnormal, and -1 means no label.

[0036] We hope to build an original log anomaly detection model. During the training process, we use the training log sequences and training labels in the training set to continuously optimize the parameters of the original log anomaly detection model to obtain the log anomaly detection model. The goal of the model training task is to optimize the parameters of the original log anomaly detection model as much as possible, so that the log anomaly detection model can detect anomalies in the log as accurately as possible.

[0037] A2. Extract the log semantic features of the training log sequence through the BERT model.

[0038] The log semantic feature extractor is used to extract the log semantic features of the training log sequence. It is the most important feature in the training log sequence and contains the semantic information in the training log sequence. An intuitive phenomenon is that if the phrases such as "fault", "error" or "shut down" appear in the training log sequence, the probability that the training log is an abnormal training log is high, which is manifested as a point anomaly. Existing research shows that pre-trained language models (such as BERT, GPT, etc.) in the field of natural language processing (NLP) have strong natural language understanding capabilities and can effectively learn text semantic feature representations. The log semantic features learned by the pre-trained language model are highly robust and can capture the contextual information in the training log sequence. Even if the training log sentences are constantly modified, the semantic features of the training log can still contain important information in the training log sequence, so that they can better adapt to the constantly changing training log output.

[0039] In natural language processing and related application fields, the performance and generalization ability of the model are key indicators to measure its effectiveness. Traditional system log anomaly detection faces many challenges in feature extraction and it is difficult to accurately capture the complex semantic information in the log. The BERT model, with its powerful bidirectional encoding ability and pre-training advantages on a large-scale corpus, can effectively mine the deep semantic features of text data. Therefore, this disclosure adopts BERT as a semantic feature extractor for the analysis of system logs.

[0040] The BERT model mainly consists of the following layers: input layer, encoder layer and output layer. Specifically: The input layer consists of: Token Embedding: The input word or subword is converted into a low-dimensional vector representation. Each word is mapped to a vector space of fixed dimension to capture the semantic information of the word.

[0041] Position Embedding: Since the Transformer model itself does not have the ability to perceive sequence position information, the position embedding layer adds a position encoding to each input word so that it can capture the sequential information in the text.

[0042] Segment Embedding: In tasks that process multiple sentences or text fragments, it is used to distinguish the embedding representations of different sentences or fragments. For example, in a text pair classification task, it can distinguish two different sentences.

[0043] Transformer Encoder Layer: Usually composed of multiple stacked encoder layers, it is the core part of the BERT model. Each encoder layer contains the following sublayers: Multi-Head Attention Layer: This layer allows the model to capture long-term dependencies in the text from different positions and different representation subspaces in parallel. It uses multiple heads of attention to simultaneously focus on different parts of the input text, calculate the degree of association between each position and other positions, and thus generate a weighted contextual representation for each position.

[0044] Layer Normalization: Normalizes the input of each layer of the neural network to accelerate model convergence, reduce gradient vanishing or exploding problems, and improve model stability and generalization capabilities.

[0045] Feed-Forward Neural Network (FFN): It consists of two fully connected layers, with activation functions such as ReLU usually used in the middle to further extract and transform the output of the attention mechanism, mapping it to a higher-dimensional feature space to increase the expressive power of the model.

[0046] The output layer consists of: Pooling Layer: There are generally multiple pooling methods, such as CLS pooling, which takes the vector representation of the first position of the input sequence (usually the [CLS] tag) as the comprehensive representation of the entire input text for subsequent classification tasks; there are also methods such as average pooling, which averages the vector representation of the sequence to obtain the overall characteristics of the text.

[0047] Classifier / Prediction Layer: Based on the output of the pooling layer, it maps the features to the corresponding task space through one or more fully connected layers and activation functions according to the specific task, such as text classification, named entity recognition, etc., and outputs the final prediction results, such as category labels, entity labels, etc.

[0048] However, during the model training process, if all parameters of the BERT model are fully trained, it is easy to cause the model to overfit, thereby reducing the generalization ability of the model. To solve this problem, the present disclosure adopts a strategy that combines parameter freezing and fine-tuning. Specifically, Figure 2 As shown in the figure, the parameters of the BERT model except the output layer are frozen. These frozen parameters retain the general language knowledge and semantic understanding capabilities learned in the pre-training stage, so that the model can analyze system logs based on the existing strong semantic understanding foundation. At the same time, only the output layer parameters are fine-tuned. The output layer is directly related to the anomaly detection task. By fine-tuning these parameters, the model can better adapt to the specific task requirements of system log anomaly detection, while maintaining general semantic understanding, while improving the accuracy of abnormal pattern recognition.

[0049] This unique parameter adjustment strategy not only avoids the overfitting problem caused by full training, but also gives full play to the advantages of the BERT model in semantic feature extraction, providing an efficient, accurate and well-generalized solution for system log anomaly detection. Compared with traditional methods, it significantly improves the performance and reliability of anomaly detection.

[0050] A3. Extract the log sequence features of the training log sequence through the Transformer encoder.

[0051] like Figure 3As shown, this step is used to extract the log sequence features of the training log sequence, which represents the execution sequence of the training log events. If there is a sequential dependency between two events, for example, event A→event B behaves normally, while event B→event A behaves abnormally. This abnormality can usually be called a contextual abnormality, and such anomalies can be detected through log sequence features. Most of the existing methods are mainly based on LSTM to extract and learn log sequence features for anomaly detection. However, the training of LSTM is serial and the computational efficiency is low. Transformer has become more popular in recent years due to its parallelism and high computational efficiency, and has performed well in log anomaly detection tasks. Therefore, the present disclosure uses a Transformer encoder pair as a log sequence feature extractor, and uses learnable position encoding and batch normalization to make it more suitable for sequence modeling. The log sequence feature extractor is divided into an embedding layer, a position encoding layer, a multi-head self-attention layer, and a feedforward layer.

[0052] Specifically, step A3 includes the following sub-steps A31-A34: A31. Input the training log sequence into the embedding layer to obtain the vector representation of each training log sequence.

[0053] First, each training log sequence Input embedding layer, which is equivalent to a lookup table, to get the vector representation corresponding to each training log sequence .

[0054] A32. Input the training log sequence into the position encoding layer to obtain the position encoding of each training log sequence.

[0055] Then, position encoding is performed on each training log sequence. Different from the original version of the Transformer encoder, the present disclosure uses a learnable position encoding method and trains the corresponding parameters of the position encoding through the back-propagation algorithm.

[0056] In natural language processing tasks, the position information of elements in a sequence is crucial for the model to understand semantics and context. The Transformer architecture performs well in processing sequence data due to its self-attention mechanism, and position encoding is a key part of it, which helps the model capture the order of elements in the sequence. Traditional Transformer encoders use a fixed position encoding method, but the present disclosure has innovated on this basis and adopted a learnable position encoding method.

[0057] The purpose of position encoding is to give each position in the input sequence a unique vector representation, so that the model can distinguish elements at different positions. The learnable position encoding adopted in the present disclosure means that the parameters of the position encoding are not fixed, but can be adjusted and optimized through training like other parameters of the model.

[0058] The back propagation algorithm is the core method for training model parameters in deep learning. It also plays a key role in the learnable position encoding disclosed in the present invention. By incorporating the position encoding parameters into the parameter system of the model, during the training process, the back propagation algorithm calculates the gradient of the position encoding parameters based on the error calculated by the loss function of the model, and then uses optimization algorithms (such as stochastic gradient descent and its variants) to update these parameters. In this way, the position encoding can be adaptively adjusted according to specific tasks and data, and better adapt to different sequence patterns and semantic information.

[0059] The original Transformer encoder uses fixed position encoding, and its position encoding mode is predefined and cannot be adjusted according to specific data sets and tasks. The learnable position encoding method disclosed in this disclosure can learn position representations that are more suitable for the current task through training. For example, in text data in certain specific fields, the positional relationship between elements may have unique patterns, and the learnable position encoding can capture these patterns, thereby improving the performance of the model.

[0060] In actual natural language processing tasks, such as text classification, machine translation, question-answering systems, etc., the learnable position encoding method disclosed in this disclosure can bring more powerful position perception capabilities to the model. Taking the text classification task as an example, the learnable position encoding can help the model better understand the location of key information in the text, so as to more accurately judge the category of the text. In the machine translation task, it can allow the model to more accurately process the order differences of elements in the source language and the target language, and improve the quality and accuracy of the translation.

[0061] A33. Input the vector representation and corresponding position encoding of each training log sequence into the multi-head self-attention layer to obtain the attention feature corresponding to each training log sequence.

[0062] Specifically, the position encoding and vector representation are added together before entering the multi-head self-attention layer. Assuming that the multi-head self-attention layer has L layers, the position encoding and vector representation of the training log sequence are added together to obtain Input to the first layer of the multi-head self-attention layer and calculated by the following formula: ; ; in, Indicates The input of the layer, Indicates The output of the layer, (1,2, … ,L), the input of the first layer is , No. The layer creates three weight matrices: Query matrix , Key matrix and value matrix. Multiplying these three matrices respectively, we get , and matrix:

[0063] Then calculate the self-attention: ; in, , , represents the dimensions of query, key, and value. For the multi-head self-attention version, assume that there are Head, Respectively represent The query, key and value of each header are Head, After self-attention of the layer, we get The final output attention feature It is obtained by concatenating all single-head attentions, namely: : in, , , .

[0064] A34. Input the attention feature corresponding to each training log sequence into the feed-forward layer to obtain the log sequence feature of each training log sequence.

[0065] The feedforward layer consists of two linear layers and the activation function GELU (Gaussian Error Linear Unit), namely: ; in, It is a linear layer, which is used to perform linear transformation. It transforms the input data through the weight matrix and bias vector.

[0066] ; in, is the weight matrix, is the bias vector, is the output vector.

[0067] The approximate calculation formula of GELU is: ; After the linear layer and activation function, the feedforward output is obtained.

[0068] The feedforward output is recorded as ,right The first dimension of is summed, and then passed through a linear layer to obtain the log sequence feature representation of the entire training log sequence ,Right now: .

[0069] A4. Input the log semantic features and the corresponding log sequence features of each training log sequence into the feature fusion module to obtain the fusion features of each training log sequence.

[0070] After the log semantics and log sequence pass through the corresponding feature extractor, the log semantic features of the training log sequence are obtained respectively. and log sequence characteristics and Then, the two features are concatenated to obtain the fusion feature .

[0071] The feature fusion module is the core of the entire process. Its main goal is to organically combine the log semantic features and log sequence features of the training log sequence to form a more representative and informative fusion feature. These two features describe the training log from different perspectives, and may not fully reflect the overall characteristics of the log when used alone. For example, in the log anomaly detection task, the fusion feature can more keenly capture the abnormal pattern in the log, whether it is a semantic abnormal expression or a sequence abnormal behavior, so as to timely discover potential problems in the system. Through fusion, the advantages of each can be fully utilized to improve the understanding and processing capabilities of the subsequent model for the training log.

[0072] A5. Input the fused features of each training log sequence into the regularization model to obtain the enhanced features of each training log sequence.

[0073] After completing the feature fusion of the training log sequence, the fused features of each training log sequence are input into the regularization model. The regularization model used in this scenario is Dropout, and the purpose is to obtain enhanced features for each training log sequence. This step plays a key role in model training and helps improve the generalization ability and stability of the model.

[0074] Dropout is a simple and effective regularization technique. Its core idea is to randomly "discard" (set the output of neurons to 0) some neurons during training. When the fused features of the training log sequence are input into the Dropout model, each neuron has a certain probability (usually recorded as the dropout rate, such as 0.5) to be randomly selected and temporarily removed from the network. This random dropout operation is different in each training iteration, which is equivalent to building a different sub-network in each training.

[0075] Without Dropout, neurons in a neural network may become overly dependent on each other, forming a complex co-adaptive relationship. This makes the model perform well on training data, but it is prone to overfitting when faced with new data. By randomly discarding neurons through Dropout, each neuron cannot be overly dependent on other specific neurons, forcing them to learn more robust and generalized features. For example, when processing a training log sequence, a neuron may only respond to a specific combination of fused features. After using Dropout, it needs to learn a more general feature pattern, so that the enhanced features of the output are more representative.

[0076] Since Dropout randomly generates different sub-networks during each training, it is equivalent to training multiple different models. The final enhanced features can be regarded as a comprehensive reflection of the learning results of these different sub-networks. This model averaging effect helps to reduce the variance of the model and improve the stability and generalization ability of the model on different data sets.

[0077] That is, the present disclosure uses Dropout for feature enhancement during the training process, which randomly closes the connections of some neurons in the neural network. If the closed neurons are different, the final output of the model will be different. In the process of log updating and evolution, log text may be deleted, added, and replaced. The updated log may also generate new log events after log parsing, which eventually causes the extracted features to be disturbed. The present disclosure uses Dropout for feature enhancement during the training process, which is similar to changing the log semantics or log sequence, so it can also be regarded as a simulation of the log evolution and update method. Based on this assumption, the feature enhancement module and the contrastive learning module can make the model more adaptable to the constantly evolving logs.

[0078] Specifically, the fusion features of the training log sequence After Dropout, enhanced features are obtained . Dropout is used to obtain enhanced features The previous layer of the neural network closes the neurons with a certain probability and obeys the Bernoulli distribution. and fusion features are positive pairs, because Is The enhancement can be regarded as Another view of . This positive sample pair can be recorded as . The other fused features in the current training log sequence are negative sample pairs, and they belong to different views.

[0079] Enhanced features contain more generalized information, allowing the model to better perform tasks such as anomaly detection and classification when faced with unseen test log sequences. For example, in system log anomaly detection, a model using enhanced features can more accurately identify different types of abnormal logs, even if these abnormal patterns have not fully appeared in the training data. Furthermore, Dropout effectively alleviates the problem of overfitting the model on the training data by generating enhanced features. The model does not over-memorize the noise and specific patterns in the training data, but learns more essential feature laws. In practical applications, this means that the model can maintain good performance during long-term use and will not fluctuate significantly due to small changes in the data.

[0080] A6. Input the fused features of each training log sequence into the probability output model to obtain the predicted abnormal log probability of each training log sequence.

[0081] The predicted abnormal log probability of each training log sequence can be obtained by the following formula: ; in, It is the probability that the model predicts the training log sequence as abnormal, that is, the probability of predicting abnormal logs, which is the fusion feature of the training log sequence The result after the linear layer and softmax.

[0082] A7. Input the enhanced features, fused features, predicted abnormal log probability and training labels of each training log sequence into the weakly supervised loss function.

[0083] The formula of the loss function of the weakly supervised form in this disclosure is: ; ; ; in, represents the loss function, represents the contrastive learning loss function, represents the cross entropy loss function; represents a hyperparameter used to balance the weight of the cross entropy loss function and the contrastive learning loss function in the total loss function; B represents the number of training log sequences in the current batch (the M training log sequences can be divided into multiple batches for training respectively. Here, B represents the number of training log sequences in the current batch. If multiple batches are not divided for training, then B here is the same as M); represents the cosine similarity between positive sample pairs, Indicates The fusion features of the training log sequences, Indicates Enhanced features of training log sequences; represents the cosine similarity between negative sample pairs, Indicates The fusion features of the training log sequences, represents the temperature hyperparameter; Indicates training log sequences; Indicates that the The predicted abnormal log probability of a training log sequence is It is obtained after being processed by the linear layer and the softmax function in sequence.

[0084] The purpose of contrastive learning is to bring similar samples (positive sample pairs) closer together in the feature space and push dissimilar samples (negative sample pairs) apart in the feature space, so that the feature representations of normal log sequences and abnormal log sequences are clearly distinguished in the feature space. The contrastive learning module performs contrastive learning on the fusion features in the training log sequence, and the calculation formula is: .

[0085] like Figure 4 As shown, for log 1, we get the log sequence of log 1 [1, 2, 1, … ], and then use the BERT model to extract its log sequence features and log semantic features, and then use the feature fusion module to fuse the log sequence features and log semantic features of log 1 to obtain the fused features of the log sequence of log 1, and then use Dropout to obtain the enhanced features of the log sequence of log 1 for log 2, and obtain the log sequence of log 2 [2, 1, 1, … ], and then extract its log sequence features and log semantic features through the BERT model, and then fuse the log sequence features and log semantic features of log 2 through the feature fusion module to obtain the fusion features of the log sequence of log 2, which is also the other fusion features in the figure; for log 3, obtain the log sequence of log 3 [2, 3, 1, …], then the log sequence features and log semantic features of log 3 are extracted through the BERT model, and then the log sequence features and log semantic features of the log sequence of log 3 are fused through the feature fusion module to obtain the fused features of the log sequence of log 3, which are the other fused features in the figure; then in the contrastive learning model, the enhanced features of the log sequence of log 1 are pulled into the fused features of the log sequence of log 1 in the feature space, and the other fused features are pushed away in the feature space, so that the feature representations of normal log sequences and abnormal log sequences are expected to be clearly distinguished in the feature space.

[0086] In addition, the present disclosure also uses cross entropy loss to perform supervised learning on a small number of labeled logs. The calculation formula is: ; in, It is training labels for training log sequences, is the probability of predicting the training log sequence as an anomaly, which is a feature The result after the linear layer and softmax is: ; Finally, the loss function is designed as a weakly supervised form, and the calculation formula is: .

[0087] In weakly supervised learning, the label information may not be completely accurate or complete, but the cross entropy loss can still guide the model to learn in the correct classification direction based on the given label information, so that the prediction result is as close as possible to the given label in probability distribution.

[0088] Cross entropy loss mainly uses the given weakly supervised label information to guide model learning from the perspective of classification, so that the model can make preliminary classification judgments on samples based on limited label information. Contrastive learning loss, on the other hand, starts from the similarities and differences of the data itself, and by comparing the features between different samples, it mines the intrinsic structure and semantic information of the data, allowing the model to learn more general and discriminative feature representations. The combination of the two can complement each other. Cross entropy loss provides a certain learning direction for the model with the help of label information, while contrastive learning loss further enhances the model's understanding of data and feature extraction capabilities in an unsupervised environment, thereby improving the performance of the model in a weakly supervised environment.

[0089] The weakly supervised loss function disclosed in the present invention can simultaneously utilize a small amount of label information (through cross entropy loss) and the inherent structural information of the data (through contrastive learning loss), fully tap the supervision signals from different sources, and improve the learning effect of the model. Moreover, the addition of contrastive learning enables the model to learn richer and more discriminative features, not just limited to simple classification feature learning based on labels, which helps to improve the generalization ability of the model and the ability to handle complex data. Furthermore, in a weakly supervised environment, labels may be noisy or incomplete. Contrastive learning loss can alleviate the impact of label noise on model training to a certain extent, because it relies more on the intrinsic relationship of the data itself for learning, rather than relying entirely on potentially inaccurate labels.

[0090] A8. Based on the loss function, the back propagation algorithm is used to continuously update and iterate the model parameters of the BERT model and the Transformer encoder to minimize the loss function. The BERT model and the Transformer encoder corresponding to the minimized loss function are the target BERT model and the target Transformer encoder. The final log anomaly detection model includes the target BERT model, the target Transformer encoder, the feature fusion module and the probability output model.

[0091] During training, the loss function is minimized as much as possible, and the trainable parameters of the entire model are updated through the back-propagation algorithm, including the output layer parameters in the BERT model and the embedding layer, position encoding layer, multi-head attention layer, and feed-forward layer in the Transformer encoder.

[0092] After the above-mentioned log anomaly detection model detects the predicted abnormal log probability of each training log sequence, it is necessary to compare the abnormal log probability with the abnormal threshold to determine whether each training log sequence is an abnormal log sequence. In the present disclosure, a method for obtaining the abnormal threshold is also proposed, including steps B1-B3: B1. Obtain the predicted abnormal log probability obtained in each iteration as a candidate value.

[0093] B2. Get the F1 score corresponding to each candidate value.

[0094] B3. Obtain the candidate value corresponding to the highest F1 score as the abnormal threshold.

[0095] Specifically, for the abnormal threshold The choice of predicting abnormal probability for all training log sequences during the model training phase As the candidate value of the abnormal threshold, then select the probability with the highest F1 score from the candidate values ​​as the abnormal threshold And save it for anomaly detection. The F1 score is defined as the harmonic mean of precision and recall, that is:

[0096] in: Indicates the number of training log sequences that have been successfully detected, that is, the number of training log sequences whose training signatures are marked as abnormal and the number of training log sequences detected as abnormal by the log anomaly detection model; Indicates the number of training log sequences that failed detection, that is, the number of training log sequences whose training signatures were marked as abnormal but detected as normal by the log anomaly detection model; Indicates the number of training log sequences that failed detection, that is, the number of training log sequences whose training labels are marked as normal but detected as abnormal by the log anomaly detection model.

[0097] Specifically, when determining TP, FN and FP, the TP, FN and FP corresponding to each candidate value can be obtained based on each candidate value, and the F1 score corresponding to each candidate value can be calculated based on the TP, FN and FP of each candidate value obtained. After obtaining the F1 scores corresponding to all candidate values, the candidate value with the highest F1 score is selected as the abnormal threshold.

[0098] After the model is trained, the trained model can also be tested through the test set. Specifically: Get a test set, which includes: a test log sequence and a test label corresponding to each test log sequence; Collecting test sets , where Indicates test log sequences, N represents the number of test log sequences, Indicates The test labels corresponding to the test log sequences, , where 0 indicates normal and 1 indicates abnormal.

[0099] The test log sequence is input into the log anomaly detection model to obtain the predicted abnormal log probability of the test log sequence; based on the predicted abnormal log probability and the abnormal threshold of the test log sequence, whether the test log sequence is an abnormal test log is detected; and the accuracy of the log anomaly detection model is determined based on the detection results and the test labels.

[0100] In the anomaly detection stage, the trained log anomaly detection model is used to judge each test log sequence in the test set. Whether there is any abnormality.

[0101] In the testing phase, the trained log anomaly detection model is used to perform log anomaly detection on the test set, and the model parameters are no longer adjusted. , when the log anomaly detection model predicts a training log sequence as an anomaly, the probability Greater than When , the training log sequence is identified as abnormal. In the present disclosure, B1-B3 can also be performed in this step.

[0102] When using the log anomaly detection model to detect the log sequence to be detected, the target BERT model extracts the log semantic features of the log sequence to be detected; the target Transformer encoder extracts the log sequence features of the log sequence to be detected; the log semantic features of the log sequence to be detected and the corresponding log sequence features are input into the feature fusion module to obtain the fusion features of the log sequence to be detected; the fusion features of the log sequence to be detected are input into the probability output model to obtain the abnormal log probability of the detected log sequence; it is determined that when the abnormal log probability is greater than the abnormal threshold, the corresponding log sequence to be detected is an abnormal log sequence; and the abnormal log sequence is output. The implementation method of each step is similar to that in the above embodiment, and will not be repeated here.

[0103] Based on the same inventive concept, the embodiment of the present application also provides a log anomaly detection device for implementing the log anomaly detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more log anomaly detection device embodiments provided below can refer to the limitations of the log anomaly detection method above, and will not be repeated here.

[0104] In an exemplary embodiment, Figure 5 As shown, a log anomaly detection device is provided, comprising: A receiving module 11 is used to receive a log sequence to be detected; An acquisition module 12 is used to input the log sequence to be detected into a log anomaly detection model to obtain the probability of abnormal logs in each of the log sequences to be detected, wherein the log anomaly detection model is obtained by training the original anomaly detection model, and a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function is used when training the original anomaly detection model; A detection module 13, configured to determine that the corresponding log sequence to be detected is an abnormal log sequence when the abnormal log probability is greater than an abnormal threshold; The output module 14 is used to output the abnormal log sequence.

[0105] As an optional implementation, the original anomaly detection model includes a BERT model, a Transformer encoder, a feature fusion module, a regularization model and a probability output model, and the device also includes: A collection module, used to collect a training set, wherein the training set includes: a training log sequence and a training label corresponding to each training log sequence; A log semantic feature extraction module, used to extract log semantic features of the training log sequence through the BERT model; A log sequence feature extraction module, used to extract log sequence features of the training log sequence through the Transformer encoder; A fusion module, used for inputting the log semantic feature of each training log sequence and the corresponding log sequence feature into the feature fusion module to obtain a fusion feature of each training log sequence; An enhancement module, used for inputting the fused features of each of the training log sequences into a regularization model to obtain enhanced features of each of the training log sequences; A probability prediction module, used for inputting the fusion features of each of the training log sequences into the probability output model to obtain the predicted abnormal log probability of each of the training log sequences; An input module, used for inputting the enhanced features, fused features, predicted abnormal log probability and training labels of each of the training log sequences into a loss function in a weakly supervised form; An iteration module is used to continuously update and iterate the model parameters of the BERT model and the Transformer encoder based on the loss function by using a back propagation algorithm to minimize the loss function, wherein the BERT model and the Transformer encoder corresponding to minimizing the loss function are a target BERT model and a target Transformer encoder, and the log anomaly detection model includes the target BERT model, the target Transformer encoder, the feature fusion module and the probability output model.

[0106] As an optional implementation, the log sequence feature extraction module is specifically used to: Input the training log sequence into the embedding layer to obtain a vector representation of each training log sequence; Input the training log sequence into a position coding layer to obtain a position coding of each training log sequence; Inputting the vector representation of each training log sequence and the corresponding position encoding into a multi-head self-attention layer to obtain an attention feature corresponding to each training log sequence; The attention feature corresponding to each of the training log sequences is input into the feed-forward layer to obtain the log sequence feature of each of the training log sequences.

[0107] As an optional implementation, the formula of the loss function of the weak supervision form is: ; ; ; in, represents the loss function, represents the contrastive learning loss function, represents the cross entropy loss function; represents a hyperparameter used to balance the weight of the cross entropy loss function and the contrastive learning loss function in the total loss function; B represents the number of training log sequences in the current batch (the M training log sequences can be divided into multiple batches for training respectively. Here, B represents the number of training log sequences in the current batch. If multiple batches are not divided for training, then B is the same as M); represents the cosine similarity between positive sample pairs, Indicates The fusion features of the training log sequences, Indicates Enhanced features of training log sequences; represents the cosine similarity between negative sample pairs, Indicates The fusion features of the training log sequences, represents the temperature hyperparameter; Indicates training log sequences; Indicates The predicted abnormal log probability of a training log sequence, For the general It is obtained after being processed by the linear layer and the softmax function in sequence.

[0108] As an optional implementation, the device further includes: A candidate value acquisition module, used to acquire the predicted abnormal log probability obtained in each iteration as a candidate value; An F1 score acquisition module, used to obtain the F1 score corresponding to each candidate value; The abnormality threshold acquisition module is used to acquire the candidate value corresponding to the highest F1 score as the abnormality threshold.

[0109] As an optional implementation, the device further includes: A test set acquisition module is used to acquire a test set, wherein the test set includes: a test log sequence and a test label corresponding to each test log sequence; A predicted abnormal log probability acquisition module is used to input the test log sequence into the log anomaly detection model to obtain the predicted abnormal log probability of the test log sequence; The detection module 13 is further used to detect whether the test log sequence is an abnormal test log according to the predicted abnormal log probability of the test log sequence and the abnormal threshold; The accuracy acquisition module is used to determine the accuracy of the log anomaly detection model according to the detection result and the test label.

[0110] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a log anomaly detection method is implemented.

[0111] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0113] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0114] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0117] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0118] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A log anomaly detection method, characterized in that: The log anomaly detection method comprises: Receive the log sequence to be detected; Inputting the to-be-detected log sequence into a log anomaly detection model to obtain the probability of abnormal logs in each of the to-be-detected log sequences, wherein the log anomaly detection model is obtained by training the original anomaly detection model, and a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function is used when training the original anomaly detection model; Determine that the log sequence to be detected corresponding to when the abnormal log probability is greater than the abnormal threshold is an abnormal log sequence; Output the exception log sequence.

2. The log anomaly detection method according to claim 1, characterized in that: The original anomaly detection model includes: a BERT model, a Transformer encoder, a feature fusion module, a regularization model and a probability output model, and the log anomaly detection method also includes: Collecting a training set, the training set comprising: a training log sequence and a training label corresponding to each training log sequence; Extracting log semantic features of the training log sequence through the BERT model; Extracting log sequence features of the training log sequence through the Transformer encoder; Inputting the log semantic features and the corresponding log sequence features of each of the training log sequences into the feature fusion module to obtain the fusion features of each of the training log sequences; Inputting the fused features of each of the training log sequences into the regularization model to obtain enhanced features of each of the training log sequences; Inputting the fusion features of each of the training log sequences into the probability output model to obtain the predicted abnormal log probability of each of the training log sequences; Inputting the enhanced features, fused features, predicted abnormal log probability and training labels of each of the training log sequences into the loss function; Based on the loss function, a back propagation algorithm is used to continuously update and iterate the model parameters of the BERT model and the Transformer encoder to minimize the loss function, wherein the BERT model and the Transformer encoder corresponding to minimizing the loss function are the target BERT model and the target Transformer encoder, and the log anomaly detection model includes the target BERT model, the target Transformer encoder, the feature fusion module and the probability output model.

3. The log anomaly detection method according to claim 2, characterized in that: The extracting the log sequence features of the training log sequence by the Transformer encoder includes: Input the training log sequence into the embedding layer to obtain a vector representation of each training log sequence; Input the training log sequence into a position coding layer to obtain a position coding of each training log sequence; Inputting the vector representation of each training log sequence and the corresponding position encoding into a multi-head self-attention layer to obtain an attention feature corresponding to each training log sequence; The attention feature corresponding to each of the training log sequences is input into the feed-forward layer to obtain the log sequence feature of each of the training log sequences.

4. The log anomaly detection method according to claim 2, characterized in that: The formula of the loss function is: ； ； ； Among them, the represents the loss function, the represents the contrastive learning loss function, the represents the cross entropy loss function; represents a hyperparameter; B represents the number of training log sequences; Indicates training log sequences; Represents the cosine similarity between positive sample pairs, Indicates the The fusion features of the training log sequences are Indicates the Enhanced features of training log sequences; Represents the cosine similarity between negative sample pairs, Indicates j The fusion features of the training log sequences are represents the temperature hyperparameter; Indicates the The predicted abnormal log probability of a training log sequence is In order to It is obtained after being processed by the linear layer and the softmax function in sequence.

5. The log anomaly detection method according to claim 4, characterized in that: The log anomaly detection method further includes: Obtaining the predicted abnormal log probability obtained in each iteration as a candidate value; Obtain the F1 score corresponding to each of the candidate values; The candidate value corresponding to the highest F1 score is obtained as the abnormal threshold.

6. The log anomaly detection method according to claim 5, characterized in that: The method further comprises: Obtain a test set, wherein the test set includes: a test log sequence and a test label corresponding to each test log sequence; Inputting the test log sequence into the log anomaly detection model to obtain the predicted abnormal log probability of the test log sequence; Detecting whether the test log sequence is an abnormal test log according to the predicted abnormal log probability of the test log sequence and the abnormal threshold; The accuracy of the log anomaly detection model is determined based on the detection result and the test label.

7. A log anomaly detection device, characterized in that: The log anomaly detection device comprises: A receiving module, used for receiving a log sequence to be detected; An acquisition module is used to input the log sequence to be detected into a log anomaly detection model to obtain the probability of abnormal logs in each of the log sequences to be detected, wherein the log anomaly detection model is obtained by training the original anomaly detection model, and a weakly supervised loss function composed of a cross entropy loss function and a contrastive learning loss function is used when training the original anomaly detection model; A detection module, used for determining that the corresponding log sequence to be detected is an abnormal log sequence when the abnormal log probability is greater than an abnormal threshold; An output module is used to output the abnormal log sequence.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the log anomaly detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the log anomaly detection method described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the log anomaly detection method described in any one of claims 1 to 6 are implemented.

Images