A log anomaly detection method and system based on prompt tuning and migration

Abstract

The application belongs to the technical field of network security, and discloses a Prompt Tuning and migration log anomaly detection method and system, which first adopts a log analysis technology based on a few-shot Prompt Tuning, uses a pre-training language model and a virtual label mark to convert log analysis into a token classification task, and can efficiently and accurately identify log templates and parameters only by using a small amount of labeled samples. Secondly, a two-stage migration learning anomaly detection framework is constructed: an encoder containing a Log-Attention module is pre-trained on source domain data; then on the target domain, the pre-training parameters are frozen, a lightweight serial adapter module is inserted, and only the adapter parameters are optimized, and finally, end-to-end anomaly detection optimization is realized by minimizing the binary cross-entropy loss. The application improves the analysis efficiency and accuracy through Prompt Tuning, and significantly enhances the generalization ability and adaptability of the anomaly detection model in cross-domain scenarios through pre-training, adapter optimization and Log-Attention mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a log anomaly detection method and system based on Prompt Tuning and migration. BACKGROUND

[0002] System logs, as the key information carrier recording system running status, user behavior and various events, play a crucial role in system monitoring, fault diagnosis, security audit and operation and maintenance management. Log analysis technology, especially log parsing and anomaly detection, is the core link to ensure stable operation of the system and network security.

[0003] Log parsing aims to automatically extract structured information, such as event templates (representing log patterns) and parameters (representing specific values of changes), from massive, usually unstructured raw log data. Accurate log parsing is the basis for subsequent log analysis tasks (such as anomaly detection). However, existing log parsing methods generally have shortcomings. Many methods rely on heuristic rules or regular expressions, which often need to be designed and maintained manually by domain experts, not only time-consuming and laborious, but also difficult to adapt to the diversity and evolution of log formats, especially when dealing with heterogeneous logs from different systems or applications, the accuracy and robustness of parsing are often unsatisfactory. In addition, most existing methods lack effective automated error correction mechanisms, making it difficult to ensure continuous efficiency and accuracy in practical applications.

[0004] Log anomaly detection aims to identify abnormal events or sequences that deviate significantly from normal behavior patterns by analyzing patterns in log sequences. These abnormalities may indicate system failures, performance bottlenecks or security attacks. Traditional anomaly detection methods usually work on parsed log template sequences. Feature extraction is a key step, common techniques such as Word2Vec, FastText and other word embedding methods can capture some semantic information, but often ignore the internal structure information of log messages and the specific position information of words in log templates, resulting in insufficient understanding of the deep semantics of log event templates. In particular, for out-of-vocabulary (OOV) words, i.e. words not in the training dictionary, the processing ability of these technologies is limited, further limiting the performance of anomaly detection models.

[0005] Furthermore, existing log anomaly detection models have significant generalization problems in cross-domain applications. Most models are trained and optimized on a specific type or source of log data (single domain), and when applied to new, unseen data sources (target domain) or system upgrades that cause log pattern changes, performance often drops sharply. This requires retraining or extensive fine-tuning on new data, which not only has high computational costs, but also limits the flexibility and efficiency of model deployment in practice. At the same time, the noise commonly present in actual log data, such as unstable event sequences or parsing errors, also interferes with the accuracy of anomaly detection, and existing methods have limited ability to handle such noise.

[0006] In summary, the current log analysis field has challenges and room for improvement in log parsing accuracy, automation, comprehensive feature representation, robustness to noise, and cross-domain adaptability. There is an urgent need for a log anomaly detection method that can efficiently and accurately parse diverse logs, fully utilize log semantic and structural information, and effectively adapt to different data domains and environmental changes. SUMMARY

[0007] In view of the deficiencies of the prior art, the present application aims to provide a log anomaly detection method and system based on Prompt Tuning and transferability. The main purpose is to provide a log anomaly detection method that can efficiently and accurately parse diverse logs, fully utilize log semantic and structural information, and effectively adapt to different data domains and environmental changes.

[0008] In a first aspect, the embodiments of the present application provide a log anomaly detection method based on Prompt Tuning and transferability, which comprises:

[0009] performing log parsing to obtain a log template sequence; and

[0010] performing transferable log anomaly detection based on the log template sequence;

[0011] The step of performing log parsing comprises:

[0012] S11: using a few-shot Prompt Tuning strategy, using a pre-trained language model to predict the tokens in the input original log message as pre-set virtual label markers or original markers of the tokens, wherein the tokens at the parameter position are predicted as the virtual label markers, and the tokens at the keyword position are predicted as the original markers;

[0013] S12: constructing a target sequence, replacing the tokens at the parameter position with virtual label markers and retaining the original tokens at the keyword position, and training the language model by maximizing the probability of the target sequence;

[0014] S13: based on the token classification prediction result, tokenizing the log message and predicting the label of each token, if it is a virtual label mark, it is classified into a parameter list, otherwise it is kept as a template content, and finally the continuous parameters are merged to generate a log template and a parameter list;

[0015] The step of performing the migration log anomaly detection comprises:

[0016] S21: pre-training the first encoder on a source domain log dataset, the first encoder comprising a Log-Attention module for introducing a bias term based on log parameter information in self-attention calculation, to obtain pre-training parameters;

[0017] S22: freezing the pre-training parameters on a target domain log dataset, and serially inserting a lightweight adapter module at a preset position of the first encoder to form a second encoder;

[0018] S23: using a log template sequence of the target domain as input, processing through the second encoder, wherein the frozen pre-training parameters remain unchanged, and only the parameters of the lightweight adapter module are updated by the gradient of the back propagation algorithm;

[0019] S24: the adapter module cooperates with the frozen pre-training parameters, inputs a new log sequence of the target domain into the second encoder for feature encoding, and finally outputs the corresponding anomaly probability prediction result.

[0020] As an optional implementation of the first aspect of the application, before step S11, the method further comprises: using an adaptive random sampling algorithm to select a small number of labeled samples from the training log messages for training the pre-training language model, the adaptive random sampling algorithm calculates the minimum distance between the candidate sample and the selected sample set in each iteration, and preferentially selects the candidate sample with the maximum minimum distance to join the selected sample set until the preset sample number is reached.

[0021] As an optional implementation of the first aspect of the application, in step S11, the Prompt Tuning strategy uses a prompt template in the form of "[X] [S] is a [MASK]" for token classification prediction, wherein [X] fills the original log message input, [S] fills the token to be identified, and [MASK] is a label slot predicted by the pre-training language model.

[0022] As an optional implementation of the first aspect of the application, in step S21, before inputting the log template sequence into the first encoder, the method further comprises: using a pre-trained Sentence-BERT model as a feature extractor to convert each session in the log template sequence into an embedded vector sequence.

[0023] As an optional implementation of the first aspect of the application, in step S21, the Log-Attention module performs self-attention operation by calculating Q, K, V are query matrix, key matrix and value matrix respectively, d is input feature dimension, h is the number of attention heads, is an additional bias term obtained from parameter encoding, P is a bias term matrix generated according to the log parameter information extracted from the corresponding parameter list.

[0024] As an optional implementation of the first aspect of the application, in step S22, the lightweight adapter module is serially inserted between the Log-Attention layer and the feedforward network layer in the first encoder, and the adapter module contains a lower projection linear layer and an upper projection linear layer, the lower projection linear layer maps the hidden layer dimension d of the input vector to a small dimension m, and the upper projection linear layer maps the small dimension m back to the dimension d.

[0025] In a second aspect, the embodiments of the application provide a log anomaly detection system based on Prompt Tuning and transferability, which comprises:

[0026] A log parsing module is configured to parse logs to obtain a log template sequence, comprising:

[0027] A log classification prediction unit is configured to use a few-shot Prompt Tuning strategy to use a pre-trained language model to classify tokens in an input original log message into pre-set virtual label markers or original markers of tokens, wherein tokens at parameter positions are predicted as the virtual label markers, and tokens at keyword positions are predicted as the original markers.

[0028] A log training unit is configured to construct a target sequence, replace tokens at parameter positions with virtual label markers and keep original tokens at keyword positions, and train a language model by maximizing the probability of the target sequence.

[0029] A log template sequence generation unit is configured to tokenize and predict token labels based on token classification prediction results, and if a token label is a virtual label marker, it is classified into a parameter list, otherwise it is kept as a template content, and finally continuous parameters are combined to generate a log template and a parameter list.

[0030] A log anomaly detection module is configured to perform transferable log anomaly detection based on the log template sequence, comprising:

[0031] An encoder pre-training unit is configured to pre-train a first encoder on a source domain log dataset, the first encoder comprising a Log-Attention module for introducing a bias term based on log parameter information in self-attention calculation, to obtain pre-training parameters;

[0032] An adapter insertion unit is configured to freeze the pre-training parameters on a target domain log dataset, and serially insert lightweight adapter modules at preset positions of the first encoder to form a second encoder.

[0033] An adapter parameter optimization unit is configured to use a log template sequence of the target domain as input and process it through the second encoder, wherein the frozen pre-training parameters remain unchanged, and only the parameters of the lightweight adapter modules are updated by a gradient algorithm.

[0034] An anomaly probability prediction unit is configured to use the adapter modules in cooperation with the frozen pre-training parameters to input a new log sequence of the target domain into the second encoder for feature encoding, and finally output corresponding anomaly probability prediction results.

[0035] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, and the program or instructions are executed by the processor to implement the steps of the method according to the first aspect.

[0036] In a fourth aspect, a readable storage medium is provided, which stores a program or instructions, and the program or instructions are executed by a processor to implement the steps of the method according to the first aspect.

[0037] Compared with the prior art, the present application has the following beneficial effects:

[0038] (1) Log parsing technology based on few-shot Prompt Tuning: The present application converts the log parsing task into a keyword and parameter prediction problem, and uses a pre-trained language model (such as RoBERTa) combined with a Prompt Tuning method of virtual label marking ("PARAM") to efficiently identify log templates and parameters with only a small number of labeled samples. This method discards the traditional dependence on regular expressions or manual rules, and significantly improves the parsing accuracy and cross-dataset adaptability through semantic-driven adaptive learning, especially solving the difficult problem of automatic processing of complex log structures.

[0039] (2) A cross-domain anomaly detection framework based on pre-training and adapter tuning: Knowledge transfer is achieved through a two-stage design (source domain pre-training + target domain adapter tuning). The Log-Attention module introduces parameter bias terms to enhance the representation of log variables during the encoding stage, while the serial adapter freezes the main model parameters through low-rank projection, and only the adapter is fine-tuned to adapt to new domains at low cost. This architecture retains the semantic knowledge of the source domain while improving the generalization ability of anomaly detection through target domain loss optimization, effectively solving the problem of the sharp performance drop of traditional methods in cross-domain scenarios.

[0040] Therefore, the log parsing method based on Prompt Tuning includes the virtual label generation, training, and online parsing process; the joint feature extraction mechanism of pre-trained models (such as Sentence-BERT) and Log-Attention encoders, especially the calculation and integration of parameter bias terms; the structural design and parameter freezing strategy in the adapter tuning stage; and the loss function design and model prediction process in cross-domain transfer. Furthermore, the adaptive sampling optimization algorithm and its application in sample selection, the integration method of the Log-Attention module and Transformer, and the implementation scheme of the end-to-end log processing system together constitute an efficient, scalable, and highly generalizable log analysis solution. Attached Figure Description

[0041] Figure 1 A flowchart of a log anomaly detection method based on Prompt Tuning and mobility provided in the first embodiment of the present invention;

[0042] Figure 2 This is a flowchart of the few-sample Prompt Tuning log parsing process in the first embodiment of the present invention;

[0043] Figure 3 This is a structural diagram of the transfer learning anomaly detection model used in the first embodiment of the present invention;

[0044] Figure 4 This is a diagram of the LogAttention structure in the first embodiment of the present invention;

[0045] Figure 5 This is a diagram of the Transformer-based serial adapter architecture in the first embodiment of the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of a log anomaly detection system based on Prompt Tuning and mobility, provided in an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0049] Example 1

[0050] Please see Figure 1 This is a flowchart of a log anomaly detection method based on Prompt Tuning and mobility proposed in the first embodiment of the present invention. The method mainly includes the following two steps:

[0051] Step 1: Perform log parsing to obtain the log template sequence.

[0052] The goal of this step is to efficiently and accurately separate the template and parameters from the raw log message to generate a log template sequence. Specifically, it includes the following sub-steps:

[0053] S11: Employing a few-shot Prompt Tuning strategy, a pre-trained language model is used to classify and predict tokens in the input raw log messages as either preset virtual labels or the original token labels. Tokens at parameter positions are predicted as virtual labels, while tokens at keyword positions are predicted as the original token labels.

[0054] Optionally, log preprocessing and adaptive sampling optimization can be performed. To improve the efficiency and accuracy of subsequent processing, the raw input log messages can be preprocessed first, such as cleaning operations, including but not limited to deleting all non-character tokens, deleting stop words, or processing camelCase.

[0055] To obtain high-quality training samples with less manual annotation work, an adaptive random sampling algorithm can be used. This algorithm selects a small number of labeled samples from the training log messages to train the pre-trained language model. The adaptive random sampling algorithm calculates the minimum distance between candidate samples and the selected sample set in each iteration, and prioritizes adding the candidate sample with the largest minimum distance to the selected sample set until a preset number of samples is reached. The specific process can be as follows:

[0056] (1) In each iteration, a set of candidate samples that do not overlap with the selected samples are randomly generated from the input domain. ;

[0057] in This represents the initially selected sample set. It is a sample set containing all available samples. Indicates sample Length measurement.

[0058] (2) Calculate the distance between each candidate sample and all samples in the selected sample set, and record the minimum distance. The calculation method is as follows: ;

[0059] in Indicates the first The sample set selected in the next iteration. It is in the The set of candidate samples available in the next iteration. Indicates candidate samples The "clean" characteristic and the selected sample set medium sample A similarity measure between the "clean" features.

[0060] (3) Select the sample with the largest minimum distance from the candidate samples (i.e., the sample farthest from the selected samples) as the best sample and add it to the selected set. Repeat this process until the sample quantity requirement is met. The final expression for the training set is: .

[0061] in This represents the final training dataset. Indicates sample The original characteristics. It indicates that it is in the first The final sample set selected in the next iteration.

[0062] In sub-step S11, through prompt-tuning and the entity-oriented language model (RoBERTa) objective, log parsing is transformed into a task of predicting the original vocabulary using keywords and predicting the virtual label "PARAM" using parameters, leveraging the characteristic that keywords are highly predictable while parameters are difficult to predict. For example...Figure 2 As shown, the specific process (virtual tag token generation) is as follows:

[0063] For log parsing, a standard prompt template consists of three unfilled slots: the input log message, the token to be identified, and the label for processing the token. A prompt template in the form "[X][S] is a [MASK]" is used for token classification prediction, where [X] is filled with the raw input log message, [S] is filled with the token to be identified, and [MASK] is the label slot predicted by a pre-trained language model. The LM (language model) then attempts to fill the label slot [MASK] with label words such as keywords or variables. Afterward, a linguist maps each predicted label word to a category of the input token. In the diagram, the language descriptor contains a set of label words for keywords "[const, keyword]" and parameters "[parameter]". By enumerating all tokens in the log message, the corresponding template and parameters can be extracted.

[0064] The Prompt Tuning method transforms the log parsing task into a parameter recognition problem that requires training with only a small number of labeled examples. Specifically, for a new dataset... Adjust the pre-trained language model This model uses PromptTuning to identify keywords and parameters in log messages. It takes an input of a raw log message consisting of n tokens. The model predicts the virtual label "PARAM" at the parameter position. For keywords, the model still predicts the original label. Formally, the model... Trained to generate output ,in:

[0065] (1)

[0066] The term "PARAM" indicates that the model recognizes the current token as a parameter, i.e., a virtual tag. Indicates the input number of the first... A token is a component of the original log message.

[0067] S12: Construct the target sequence, replace the tokens at the parameter positions with virtual labels and retain the original tokens at the keyword positions, and train the language model by maximizing the probability of the target sequence.

[0068] Given input log message Construct the target sequence The method is to position The parameters at each position are replaced with virtual labels “PARAM”, and the original words at the keyword positions are maintained using formula (1). Then, a language model is trained to maximize the probability of the target sequence Y. :

[0069] (2)

[0070] in It is the number of labeled training samples. Indicates the first The index of a log message, representing a single log message. The number of tokens, Indicates the first in the log message Index of each token It is the first in the target sequence Each token is generated according to formula (1).

[0071] It is important to note that the entire pre-trained model is reused during the tuning process. The entity-oriented goal is similar to the language model-based goal (i.e., masked label prediction). This can narrow the gap between pre-training and fine-tuning, allowing the model to retain the knowledge learned by the pre-trained language model.

[0072] S13: Based on the token classification prediction results, the log message is segmented and each token tag is predicted. If it is a virtual tag, it is included in the parameter list; otherwise, it is retained as template content. Finally, the continuous parameters are merged to generate the log template and parameter list, achieving efficient single-pass parsing.

[0073] Specifically, the model first segments the input into groups of words, then predicts their corresponding target words. If a word is predicted as "PARAM", it is integrated into the parameter list; otherwise, it remains in the log template. Finally, the log template is post-processed, replacing consecutive parameters with single parameters. Only one decoding process is needed to parse a single log message, which is highly efficient when processing large volumes of logs.

[0074] Step 2: Detect migration log anomalies based on log template sequences.

[0075] Build a log anomaly detection model that can adapt to different domains (datasets). Employ a transfer learning strategy that combines pre-training with adapter tuning. Specifically, this includes the following sub-steps:

[0076] S21: On the source domain log dataset, the first encoder is pre-trained, which includes a Log-Attention module for introducing a bias term based on log parameter information in the self-attention computation, to obtain pre-trained parameters.

[0077] Optionally, before inputting the log template sequence into the first encoder, a pre-trained Sentence-BERT model is used as a feature extractor to convert each session in the log template sequence into an embedding vector sequence. It is assumed that each session has a fixed length. ,enter The embedding is obtained after passing through the feature extractor:

[0078] (3)

[0079] Where FE is the encoder of Sentence-BERT, and the obtained... belong , indicating the embedding of each session.

[0080] Furthermore, the encoding after feature extraction The data is then passed to the Log-Attention encoder (the first encoder). At this stage, the Log-Attention encoder is without an adapter; it's a modified Transformer encoder used to process information from the log sequence. The Log-Attention encoder consists of multiple Transformer layers, each including a self-attention mechanism and a feedforward neural network. Figure 3 The Log-Attention module is specifically designed to encode parameters in the logs to preserve parameter information. Unlike standard self-attention mechanisms, Log-Attention adds additional bias terms to the original calculation method. These bias terms are derived from the parameter information in the logs. This bias term provides more information in the self-attention calculation through Log-Attention, helping the model understand the variables and different structures in the logs. Equation (4) is as follows:

[0081] (4)

[0082] Where Q, K, and V represent query, key, and value, respectively. This is an additional bias term obtained from the parameter encoding, where h is the number of heads in the multi-head attention and d is the dimension of the input features. Then, the self-attention output Z of each layer is passed to the feedforward neural network (FFN). The FFN consists of two linear transformations and a ReLU activation function. After linear transformation and bias activation operations on the weight matrix, the output is then subjected to layer normalization.

[0083] In the anomaly detection pre-training process, no adapter is used. The Log-Attention encoder encodes the input features solely through a self-attention mechanism and a feedforward network. The feature encoding process relies on the self-attention mechanism to focus on important input information, and parameter encoding enhances the representational power of the log information. The final output feature representation can be used for subsequent classification tasks in the target domain, i.e., binary classification of log sequences as normal or abnormal.

[0084] S22: On the target domain log dataset, freeze the pre-trained parameters and insert lightweight adapter modules serially at the preset positions of the first encoder to form the second encoder.

[0085] It's important to note that the adapter tuning phase leverages the knowledge gained during pre-training by inserting a lightweight adapter. When tuning in the target domain, only the adapter parameters are updated, while the parameters of the Log-Attention and feedforward layers are frozen. This approach avoids the high cost of fine-tuning the entire model. Figure 4 As shown, the adapter's workflow is as follows:

[0086] (1) Insert the adapter

[0087] like Figure 5 As shown, the adapter is inserted sequentially between the Log-Attention layer and the Feedforward Network (FFN) layer. The adapter itself consists of two linear transformation layers: the first layer maps the hidden layer dimension d to a smaller dimension m, and the second layer maps it back to the original d dimension. The core of the adapter is to limit the number of newly added trainable parameters through a low-rank transformation. The adapter's output... Calculated using the following formula (5):

[0088] (5)

[0089] in: It is the hidden vector passed to the adapter. It is the downward projection matrix. is the upward projection matrix, and m is the smaller dimension of the adapter. It is the output of the adapter, which ultimately serves as the input to the next layer.

[0090] (2) Freeze source domain parameters

[0091] During adapter tuning, the parameters of the Log-Attention layer and the Feedforward Neural Network (FFN) layer (i.e., the parameters of the source domain) are frozen; only the adapter parameters are trained. At this time, the Log-Attention encoder processes the data based on the knowledge learned during the pre-training phase, instead of training the entire model, and only updates the adapter parameters in the target domain.

[0092] S23: The log template sequence of the target domain is used as input and processed by the second encoder, where the frozen pre-trained parameters remain fixed and the parameters of the lightweight adapter module are updated only through the backpropagation algorithm.

[0093] During the target domain tuning phase, after passing through the Log-Attention encoder (second encoder) and adapter, the model outputs a probability value representing the probability that the input log belongs to the anomaly class (label 1). Specifically, the model output is shown in formula (6):

[0094] and (6)

[0095] in x It is the sequence of input logs in the target domain. These are the frozen parameters (parameters of the Log-Attention and FFN layers) passed down from the pre-training stage. These are the adapter parameters trained on the target domain. These two probability values ​​represent the input logs, respectively. x The probability of belonging to an anomaly (1) or normal (0). Finally, in the binary classification task, the binary cross-entropy loss (BCELoss) is used to calculate the difference between the predicted result and the actual label. For each log sequence x The formula for calculating cross-entropy loss (7) is:

[0096] (7)

[0097] Where: N is the number of samples in the target domain dataset. y i It is the first i The model is fine-tuned on the target domain by minimizing the binary cross-entropy loss. The smaller the loss, the closer the model's prediction is to the true label, meaning the model can better distinguish between normal and abnormal binary classification.

[0098] S24: The adapter module works in conjunction with the frozen pre-trained parameters to input the new log sequence of the target domain into the second encoder for feature encoding, and finally outputs the corresponding anomaly probability prediction result.

[0099] After training, the model can use log data from the target domain. x new Output the probability that it belongs to the anomaly category. P(y = 1 | x new If the probability P exceeds a preset threshold, the model classifies the log as an abnormal log; otherwise, it classifies it as a normal log.

[0100] In summary, the first embodiment above specifically includes the following two points:

[0101] First, improvements in log parsing technology have significantly increased the efficiency of log data processing. Traditional log parsing methods often rely on manually defined rules or regular expressions, which not only increases manual workload but also makes it difficult for log parsing systems to adapt to different types and formats of log data. By adopting the few-sample Prompt Tuning method, the system can automatically identify templates and parameters in logs using only a small number of labeled log samples. This method does not rely on complex manual rules and can quickly learn and adapt to the semantic information of different log data, greatly improving the flexibility and accuracy of log parsing. Furthermore, the log parsing process can accurately identify keywords and parameters in logs and accurately classify them as templates and parameters, significantly improving the automation level of data processing and reducing the need for manual intervention.

[0102] Secondly, regarding log anomaly detection, the proposed framework addresses the generalization problem in cross-domain applications by combining a pre-trained model with transfer learning. Traditional log anomaly detection systems often perform poorly in different domains or new environments, struggling to handle the diversity of different log samples. By pre-training in the source domain and optimizing with a target domain-specific adapter, the system not only retains the semantic information of the source domain but also effectively transfers to new domains, thereby improving anomaly detection capabilities in new environments. In particular, the use of the Log-Attention module enables the model to better understand the complex variables and structures in the logs during the encoding process, further improving the detection accuracy of anomalous logs.

[0103] The combination of these technologies enables the log parsing and anomaly detection method of the present invention to exhibit strong robustness and adaptability in practical applications, effectively dealing with variable log data and complex detection scenarios.

[0104] Example 2

[0105] Please see Figure 6 The diagram shows a schematic of a log anomaly detection system based on Prompt Tuning and mobility proposed in the second embodiment of this application. The system includes the following key modules:

[0106] Log parsing module 100 is used to parse logs and obtain log template sequences, including:

[0107] The log classification prediction unit 101 is used to adopt a few-shot Prompt Tuning strategy and utilize a pre-trained language model to classify and predict tokens in the input raw log message as preset virtual labels or original token labels, wherein tokens at parameter positions are predicted as virtual labels and tokens at keyword positions are predicted as original token labels.

[0108] Log training unit 102 is used to construct the target sequence, replace the tokens at the parameter positions with virtual labels and retain the original tokens at the keyword positions, and train the language model by maximizing the probability of the target sequence.

[0109] The log template sequence unit 103 is used to segment the log message and predict each token tag based on the token classification prediction result. If it is a virtual tag, it is included in the parameter list; otherwise, it is retained as template content. Finally, the continuous parameters are merged to generate the log template and parameter list.

[0110] Log anomaly detection module 200, used for detecting migratory log anomalies based on the log template sequence, including:

[0111] The encoder pre-training unit 201 is used to pre-train a first encoder on a source domain log dataset. The first encoder includes a Log-Attention module for introducing a bias term based on log parameter information in the self-attention computation to obtain pre-trained parameters.

[0112] The adapter insertion unit 202 is used to freeze the pre-trained parameters on the target domain log dataset and serially insert lightweight adapter modules at preset positions of the first encoder to form a second encoder.

[0113] The adapter parameter optimization unit 203 is used to take the log template sequence of the target domain as input and process it through the second encoder, wherein the frozen pre-trained parameters remain fixed and the parameters of the lightweight adapter module are updated only through the backpropagation algorithm.

[0114] The anomaly probability prediction unit 204 is used for the coordinated operation of the adapter module and the frozen pre-trained parameters. It inputs the new log sequence of the target domain into the second encoder for feature encoding and finally outputs the corresponding anomaly probability prediction result.

[0115] The log anomaly detection system based on Prompt Tuning and mobility in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not specifically limit the specific implementation.

[0116] The log anomaly detection system based on Prompt Tuning and portability in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0117] This application provides a log anomaly detection system based on Prompt Tuning and mobility, which can achieve... Figure 1 The various processes implemented in the method embodiment, which is based on Prompt Tuning and mobility-based log anomaly detection method, will not be described again here to avoid repetition.

[0118] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of a log anomaly detection method based on Prompt Tuning and mobility, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0119] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of a log anomaly detection method based on Prompt Tuning and mobility, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0120] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0121] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0123] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A log anomaly detection method based on Prompt Tuning and portability, characterized in that, Includes the following steps: Perform log parsing to obtain the log template sequence; as well as Migratory log anomaly detection is performed based on the log template sequence; The steps for performing log parsing include: S11: Using a few-shot Prompt Tuning strategy, a pre-trained language model is used to classify and predict tokens in the input raw log message as preset virtual labels or the original labels of the tokens. Tokens at parameter positions are predicted as the virtual labels, and tokens at keyword positions are predicted as the original labels. S12: Construct the target sequence, replace the tokens at the parameter positions with virtual labels and retain the original tokens at the keyword positions, and train the language model by maximizing the probability of the target sequence; S13: Based on the token classification prediction results, segment the log message and predict each token tag. If it is a virtual tag, it is included in the parameter list; otherwise, it is retained as template content. Finally, merge the continuous parameters to generate the log template and parameter list. The steps for performing migration log anomaly detection include: S21: On the source domain log dataset, pre-train the first encoder, which includes a Log-Attention module for introducing a bias term based on log parameter information in the self-attention computation, to obtain pre-trained parameters. S22: On the target domain log dataset, freeze the pre-trained parameters, and serially insert lightweight adapter modules at preset positions of the first encoder to form a second encoder; S23: The log template sequence of the target domain is used as input and processed by the second encoder, wherein the frozen pre-trained parameters remain fixed and the parameters of the lightweight adapter module are updated only through the backpropagation algorithm. S24: The adapter module and the frozen pre-trained parameters work together to input the new log sequence of the target domain into the second encoder for feature encoding, and finally output the corresponding anomaly probability prediction result.

2. The method according to claim 1, characterized in that, Prior to step S11, the method further includes: An adaptive random sampling algorithm is used to select a small number of labeled samples from the training log messages to train the pre-trained language model. The adaptive random sampling algorithm calculates the minimum distance between the candidate sample and the selected sample set in each iteration and prioritizes the candidate sample with the largest minimum distance to be added to the selected sample set until the preset number of samples is reached.

3. The method according to claim 2, characterized in that, In step S11, the Prompt Tuning strategy uses a prompt template of the form "[X][S] is a [MASK]" to perform token classification prediction, where [X] is filled with the original log message input, [S] is filled with the token to be identified, and [MASK] is the label slot predicted by the pre-trained language model.

4. The method according to claim 1, characterized in that, In step S21, before inputting the log template sequence into the first encoder, the method further includes: A pre-trained Sentence-BERT model is used as a feature extractor to convert each session in the log template sequence into an embedding vector sequence.

5. The method according to claim 1 or 4, characterized in that, In step S21, the Log-Attention module calculates... To perform self-attention operations, where Q, K, and V are the query matrix, key matrix, and value matrix, respectively, d is the input feature dimension, and h is the number of heads in the multi-head attention mechanism. It is an additional bias term obtained from the parameter encoding, and P is a bias term matrix generated based on the log parameter information extracted from the corresponding parameter list.

6. The method according to claim 1, characterized in that, In step S22, the lightweight adapter module is serially inserted between the Log-Attention layer and the feedforward network layer in the first encoder, and the adapter module includes a down-projection linear layer and an up-projection linear layer. The down-projection linear layer maps the hidden layer dimension d of the input vector to a small dimension m, and the up-projection linear layer maps the small dimension m back to dimension d.

7. A log anomaly detection system based on Prompt Tuning and portability, characterized in that, The system includes: The log parsing module is used to parse logs and obtain log template sequences, including: The log classification prediction unit is used to adopt a few-shot Prompt Tuning strategy and utilize a pre-trained language model to classify and predict tokens in the input raw log message as preset virtual labels or original token labels. Tokens at parameter positions are predicted as virtual labels, and tokens at keyword positions are predicted as original token labels. The log training unit is used to construct the target sequence, replace the tokens at the parameter positions with virtual labels and retain the original tokens at the keyword positions, and train the language model by maximizing the probability of the target sequence. The log template sequence unit is generated to segment the log message and predict each token tag based on the token classification prediction result. If it is a virtual tag, it is included in the parameter list; otherwise, it is retained as template content. Finally, the continuous parameters are merged to generate the log template and parameter list. The log anomaly detection module is used to perform migration log anomaly detection based on the log template sequence, including: The encoder pre-training unit is used to pre-train the first encoder on the source domain log dataset. The first encoder includes a Log-Attention module for introducing a bias term based on log parameter information in the self-attention computation to obtain pre-trained parameters. An adapter insertion unit is used to freeze the pre-trained parameters on the target domain log dataset and serially insert lightweight adapter modules at preset positions of the first encoder to form a second encoder. The adapter parameter optimization unit is used to take the log template sequence of the target domain as input and process it through the second encoder, wherein the frozen pre-trained parameters remain unchanged, and the parameters of the lightweight adapter module are updated only through the backpropagation algorithm. The anomaly probability prediction unit is used for the coordinated operation of the adapter module and the frozen pre-trained parameters. It inputs the new log sequence of the target domain into the second encoder for feature encoding and finally outputs the corresponding anomaly probability prediction result.

8. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of a log anomaly detection method based on Prompt Tuning and mobility as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, they implement the steps of the log anomaly detection method based on Prompt Tuning and mobility as described in any one of claims 1-6.

Images