ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning and related device

Abstract

The application provides an ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning and a related device, and relates to the technical field of log anomaly detection. Raw log data is obtained and converted into target log sequences through log preprocessing; a preset base model is obtained, and the preset base model is adjusted through a low-rank linear subspace representation fine-tuning technology to generate a target model; the target log sequence is input into the target model to determine a feature vector; and a preset linear classifier is used to perform log anomaly detection according to the feature vector. By intervening in the representation of known encoding specific semantic information, the low-rank linear subspace representation fine-tuning method can deeply reveal the way in which the model processes and understands data, reduce the cost of model training, and improve the technical effect of model interpretability.

Description

Technical Field

[0001] The present application relates to the technical field of log anomaly detection, and in particular to an ESN log anomaly detection method and related devices based on low-rank linear subspace representation fine-tuning. Background Art

[0002] To ensure the stability and reliability of the ESN system, log anomaly detection technology is used to monitor system logs. This technology can quickly identify and warn of potential issues, enabling the operations team to respond promptly and effectively reduce the risk of service interruptions. Currently, the mainstream log anomaly detection process consists of three stages: using a log parser to extract log templates and convert semi-structured log data into structured log data; then, segmenting the continuous log data into log sequences and constructing feature representations of the log sequences; finally, using the extracted features for anomaly detection. Log anomaly detection methods can be categorized based on the technology used: machine learning-based log anomaly detection methods, deep learning-based log anomaly detection methods, and pre-trained model-based log anomaly detection methods.

[0003] Machine learning-based methods face challenges such as difficult feature extraction, complex preprocessing steps, and high sensitivity to feature selection. Furthermore, these methods are limited in capturing the time series characteristics of log events. In contrast, deep learning-based methods can more effectively capture long-range dependencies and semantic information, but they face challenges in model complexity, training and inference time, and are prone to overfitting on small datasets. Furthermore, these models often lack transparency and interpretability, making them difficult to meet when in-depth analysis of anomaly root causes is required. When pre-trained models are applied to log anomaly detection scenarios, they demonstrate strong detection accuracy and generalization capabilities. However, fine-tuning these models to specific tasks still requires significant computational resources and time, and may encounter performance bottlenecks in certain tasks. While parameter-efficient fine-tuning methods can reduce fine-tuning costs to some extent, this approach can compromise model performance, further impacting anomaly detection accuracy.

[0004] Therefore, how to reduce model training costs and improve model interpretability during log anomaly detection has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] In order to reduce the model training cost and improve the model interpretability during the log anomaly detection process, the present application provides an ESN log anomaly detection method and related devices based on low-rank linear subspace representation fine-tuning.

[0006] In the first aspect, the present application provides an ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning, which adopts the following technical solutions:

[0007] An ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning includes:

[0008] Obtaining original log data and converting the original log data into a target log sequence through log preprocessing;

[0009] Obtaining a preset base model, and adjusting the preset base model through a low-rank linear subspace representation fine-tuning technique to generate a target model;

[0010] inputting the target log sequence into the target model to determine a feature vector;

[0011] Performing log anomaly detection based on the feature vector using a preset linear classifier;

[0012] The preset base model is the RoBERTa-base model.

[0013] Optionally, the step of obtaining original log data and converting the original log data into a target log sequence by log preprocessing includes:

[0014] Get raw log data;

[0015] Parsing the raw log data into a set of log templates using the log parser Drain, and assigning an event ID to each log template in the set of log templates;

[0016] Performing conversion according to the log template set to generate a structured log set;

[0017] The target log sequence is generated by cutting the log set in a sliding window or session manner.

[0018] Optionally, the preset base model includes: a RoBERTa-base model.

[0019] Optionally, the step of adjusting the preset base model by using a low-rank linear subspace representation fine-tuning technique to generate a target model includes:

[0020] Use the learned linear projection to adjust the representation of the model in the subspace,

[0021] Where {R, W, b} are learning parameters, while the original model parameters remain unchanged, h is the hidden representation of the applied intervention layer, is a low-rank matrix with orthogonal rows, and are the linear projection matrix and bias vector respectively;

[0022] The low-rank linear subspace representation fine-tuning technique is applied to the first two and last two bits of the output representation of the 12th attention layer of the base model RoBERTa-base to generate a target model with LoReFT intervention.

[0023] Optionally, the step of inputting the target log sequence into the target model to determine the feature vector includes:

[0024] Input the target log sequence X into the target model so that after the data is processed by Embedding and L-layer Transformer Enconder, the feature vector T of all log events in the log sequence is obtained;

[0025] T=Trm L (X')

[0026] Where X' is X after embedding.

[0027] Optionally, after the step of obtaining a preset base model and adjusting the preset base model by a low-rank linear subspace representation fine-tuning technique to generate a target model, the method further includes:

[0028] A class weight mechanism is introduced, and the target model is trained using a cross-entropy loss function with class weights;

[0029] The class weight w of class i i The calculation is:

[0030]

[0031] Where n is the total number of samples in the dataset, k is the total number of categories, and n i is the number of samples in the i-th class;

[0032] The cross entropy loss function with class weights is:

[0033]

[0034] Where k is the total number of categories, w i is the weight of the i-th class, y i is the true label of the i-th category, p i is the probability of the i-th class predicted by the model.

[0035] Optionally, before the step of performing log anomaly detection according to the feature vector using a preset linear classifier, the method further includes:

[0036] Constructing a preset linear classifier, using a hyperbolic tangent function to approximate a standard GELU as an activation function of the preset linear classifier;

[0037]

[0038] Where x is the input vector.

[0039] In a second aspect, the present application provides an ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning, the ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning comprising:

[0040] A data acquisition module is used to acquire original log data and convert the original log data into a target log sequence through log preprocessing;

[0041] A model generation module is used to obtain a preset base model and adjust the preset base model through a low-rank linear subspace representation fine-tuning technique to generate a target model;

[0042] a feature vector module, configured to input the target log sequence into the target model to determine a feature vector;

[0043] The anomaly detection module is used to perform log anomaly detection based on the feature vector using a preset linear classifier.

[0044] In a third aspect, the present application provides a computer device, comprising: a memory and a processor, wherein the processor executes the method described above when running computer instructions stored in the memory.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0046] In summary, this application has the following beneficial technical effects:

[0047] This application obtains raw log data and converts it into a target log sequence through log preprocessing; obtains a preset base model and adjusts the preset base model through low-rank linear subspace representation fine-tuning technology to generate a target model; inputs the target log sequence into the target model to determine a feature vector; and performs log anomaly detection based on the feature vector using a preset linear classifier. By intervening in the representation of known encoding-specific semantic information, the low-rank linear subspace representation fine-tuning method can deeply reveal how the model processes and understands data, achieving the technical effect of reducing model training costs and improving model interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0049] Figure 2 This is a flow chart of an embodiment of an ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning of the present application.

[0050] Figure 3 This is the data segmentation flow chart of the ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning in this application;

[0051] Figure 4 This is the model structure diagram of the ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning in this application;

[0052] Figure 5 This is a schematic diagram of the representation fine-tuning method in the ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning in this application;

[0053] Figure 6 This is a structural block diagram of an embodiment of the ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0056] like Figure 1As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0057] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0058] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an ESN log anomaly detection program based on low-rank linear subspace representation fine-tuning.

[0059] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this application can be set in the computer device, and the computer device calls the ESN log anomaly detection program based on low-rank linear subspace representation fine-tuning stored in the memory 1005 through the processor 1001, and executes the ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning provided in the embodiment of the present application.

[0060] The embodiment of the present application provides an ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning, referring to Figure 2 , Figure 2 This is a flowchart of an embodiment of an ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning of the present application.

[0061] In this embodiment, the ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning includes the following steps:

[0062] Step S10: Obtain original log data and convert the original log data into a target log sequence through log preprocessing.

[0063] It is understood that the terms in this embodiment are explained as follows:

[0064] Log parser: A tool or algorithm used to convert unstructured or semi-structured log data into a structured format. It extracts key information (such as timestamp, host name, log level, log message, etc.) and organizes it into standard structured data for subsequent analysis and processing.

[0065] Log template: A standardized format extracted from a log message, typically containing a fixed set of fields and variables. Each log template represents a class of similar log messages.

[0066] Log sequence: refers to a series of log messages or log templates arranged in chronological order. Each log sequence can be regarded as a time series data.

[0067] Feature representation: This refers to converting log data into numerical vectors or matrices that can be used in machine learning or deep learning models. These features can include event IDs, timestamps, host names, and other features of the log template.

[0068] Anomaly detection refers to the process of identifying data points or events that do not conform to normal patterns within a large amount of data. In log anomaly detection, the goal is to identify log entries that do not conform to normal log patterns.

[0069] Parameter Efficient Fine-tuning (PEFT): By adding specific network layers to the pre-trained model, only the parameters of these newly added layers are updated while the original parameters of the pre-trained model are frozen, thereby significantly reducing the number of training parameters.

[0070] Sliding window: A method for partitioning continuous data into fixed-length segments. The window can be moved from data point to data point to form multiple overlapping or non-overlapping segments.

[0071] Session: A group of related log entries, typically representing a complete interaction or operation.

[0072] Distributed Interchange Intervention (DII): A method for exploring model behavior by replacing the model's internal representations to enhance model interpretability.

[0073] Counterfactual source representation: refers to the representation produced by the model for a certain input under intervention conditions, which is used to compare and analyze the behavior of the model.

[0074] Distributed Alignment Search (DAS): An optimization method for finding the best low-rank projection matrix such that the model behaves as expected after intervention.

[0075] Linear projection matrix: A matrix used to project a high-dimensional vector into a low-dimensional space, often used for feature dimensionality reduction and representation adjustment.

[0076] Bias vector: A constant vector added to the linear transformation to adjust the output of the model.

[0077] LoRA: A parameter-efficient fine-tuning method that reduces the number of parameters that need to be trained via low-rank decomposition.

[0078] In the specific implementation, since the original log data is a semi-structured text message, it needs to be converted into a form acceptable to the model through log preprocessing. Log preprocessing includes two steps: log parsing and sequence segmentation. First, the log parser Drain is used to parse the original log into log templates and assign an event ID to each log template. Then, the log data is converted into structured data according to the log template, and each log corresponds to a log template. Finally, the log data is segmented using a sliding window or session to obtain a log sequence. The data segmentation process is as follows: Figure 3 shown.

[0079] It should be noted that the steps of obtaining original log data and converting the original log data into a target log sequence through log preprocessing include: obtaining original log data; using the log parser Drain to parse the original log data into a log template set, and assigning an event ID to each log template in the log template set; converting according to the log template set to generate a structured log set; and cutting in the log set through a sliding window or session to generate a target log sequence.

[0080] Step S20: Obtain a preset base model, and adjust the preset base model through low-rank linear subspace representation fine-tuning technology to generate a target model.

[0081] It is understandable that the preset base model includes: RoBERTa-base model.

[0082] In the specific implementation, the structure of the constructed log anomaly detection model RoBERTa_Classifier is as follows Figure 4As shown in the figure, the base model uses RoBERTa, a pre-trained language model developed by Facebook AI based on BERT. RoBERTa undergoes multiple improvements during pre-training to address potential undertraining issues with the original BERT. These improvements include using larger batch sizes, more training data, and optimized training methods, resulting in RoBERTa possessing stronger representation and generalization capabilities than BERT. The RoBERTa-base model consists of a 12-layer TransformerEncoder with a total of 125 million parameters. This choice ensures the model has strong feature extraction and anomaly detection capabilities when processing log data.

[0083] It should be noted that this embodiment is inspired by interpretability research, especially the exploration of techniques for representing causal relationships within deep learning models, such as the exchange intervention method in the causal abstraction framework. In order to determine whether a concept is encoded in the linear subspace of the model, distributed exchange intervention (DII) is introduced. Suppose that when the model processes input b, the hidden representation created at row i and column k is b, and when the same model processes input s, the corresponding representation is s. Given a counterfactual source representation s, the intervention on b can be defined as:

[0084] DII(b,s,R)=b+R T (Rs-Rb)

[0085] in is a low-rank, row-orthogonal projection matrix, d is the dimension of the representation, and r is the dimension of the subspace in which the intervention is performed. Distributed alignment search (DAS) is used to optimize this subspace R, making the model more likely to produce the expected counterfactual output after the intervention. This approach not only enhances model interpretability but also potentially becomes a powerful tool for controlling and adjusting model behavior.

[0086] In a specific implementation, the steps of adjusting the preset base model to generate the target model by using the low-rank linear subspace representation fine-tuning technology include: adjusting the representation of the model in the subspace using the learned linear projection, φ LoReFT (h)=h+R T (Wh+b-Rh):

[0087] Where {R, W, b} are learning parameters, while the original model parameters remain unchanged, h is the hidden representation of the applied intervention layer, is a low-rank matrix with orthogonal rows, and are the linear projection matrix and bias vector respectively; the low-rank linear subspace representation fine-tuning technique is applied to the first 2 and last 2 bits of the output representation of the 12th attention layer of the base model RoBERTa-base to generate a target model with LoReFT intervention.

[0088] In a specific implementation, the schematic diagram of the characterization fine-tuning method is as follows Figure 5 As shown, the left figure depicts an intervention I: the intervention function φ is applied to the hidden representation at position P in the Lth layer, and the right figure shows the intervention function used in LoReFT, which finds an edit vector that only modifies the representation in the linear subspace formed by the rows of the matrix R. Specifically, it shows how LoReFT with r = 2 works on the 3D hidden representation.

[0089] It's important to note that due to the significant imbalance between normal and abnormal data in log sequences, most existing methods use normal data to train models to learn normal patterns. They then detect abnormal logs by setting thresholds, calculating reconstruction errors, or calculating anomaly scores. These solutions often require manual setting of hyperparameters such as thresholds. Unlike existing methods, this proposal introduces a class-weighted mechanism and trains the model using a cross-entropy loss function with class weights. This allows the model to learn both normal and abnormal patterns, reducing the impact of human factors on model performance.

[0090] In a specific implementation, after obtaining a preset base model and adjusting the preset base model through a low-rank linear subspace representation fine-tuning technique to generate a target model, the following steps are also included: introducing a category weight mechanism and using a cross-entropy loss function with category weights to train the target model; the category weight w of the i-th category i The calculation is:

[0091]

[0092] Where n is the total number of samples in the dataset, k is the total number of categories, and n i is the number of samples in class i; the cross entropy loss function with class weights is:

[0093]

[0094] Where k is the total number of categories, w i is the weight of the i-th class, y i is the true label of the i-th category, p i is the probability of the i-th class predicted by the model.

[0095] Step S30: Input the target log sequence into the target model to determine the feature vector.

[0096] Step S40: performing log anomaly detection based on the feature vector using a preset linear classifier.

[0097] It should be noted that before the step of performing log anomaly detection based on the feature vector using the preset linear classifier, the following steps are also included: constructing the preset linear classifier and using the hyperbolic tangent function to approximate the standard GELU as the activation function of the preset linear classifier;

[0098]

[0099] Where x is the input vector.

[0100] It can be understood that in this embodiment, the computational cost and memory consumption required for fine-tuning the pre-trained model are further reduced, while the transparency and explainability of the model decisions are enhanced. First, the log parsing algorithm Drain is used to convert semi-structured log data into structured data; then, the continuous log data is divided into log sequences in a session or window manner; then, the low-rank linear subspace representation fine-tuning technique is used to intervene in the hidden representation calculated by the RoBERTa model during the forward pass of the attention layer, while freezing the parameters of the rest of the RoBERTa model; finally, the RoBERTa model with low-rank linear subspace intervention is fine-tuned using the log sequence to adapt it to the task of extracting log sequence feature representations, and a classifier is trained to use the [CLS] tag in the hidden state of the last layer of the RoBERTa model as the representation of the entire sequence for classification. The method of this embodiment requires 10 to 50 times fewer parameters to be trained than the most advanced parameter-efficient fine-tuning method (LoRA) without almost losing model performance, thereby significantly saving the resources and time required for model training. Furthermore, by intervening in representations known to encode specific semantic information, low-rank linear subspace representation fine-tuning methods can reveal insights into how models process and understand data, thereby building more transparent and trustworthy AI systems.

[0101] In the specific implementation, the experimental data and related instructions corresponding to this embodiment are as follows: In order to verify the effectiveness of the method and model of this embodiment in log anomaly detection, the open source Thunderbird dataset is selected to test the effect of the RoBERTa Classifier model after fine-tuning the low-rank linear subspace representation. The Thunderbird dataset is a large log dataset collected from a supercomputer system. There are 16,601,745 log data in total, of which 166,011,91 are normal logs and 554 are abnormal data. A sliding window with a size of 2 minutes and a step size of 1 minute is used to split the data, resulting in 61,339 log sequences, of which 60,795 are normal log sequences and 544 are abnormal log sequences. The average log sequence length is 1,412. 60% of the data is used as the training set, 20% of the data is used as the validation set, and 20% of the data is used as the test set. The final test results are shown in Table 1 below:

[0102] Table 1. Test results of Thunderbird dataset

[0103]

[0104] It should be noted that the technical effects and advantages of this embodiment are:

[0105] In this embodiment, after applying r=2 intervention to the base model RoBERTa-base for log anomaly detection using the low-rank subspace representation fine-tuning method, the trainable parameters are only 3,074, accounting for 0.0025% of the total model parameters, greatly saving the resources and time required for model training. Moreover, this intervention is relatively controllable and can enhance the interpretability of the model to a certain extent.

[0106] This embodiment introduces a category weight mechanism to solve the problem of severe imbalance in the number of normal and abnormal logs and improve the stability of model training. Experimental testing shows that the RoBERTaClassifier model, which uses the low-rank subspace representation fine-tuning method, performs three rounds of training on the Thunderbird dataset and achieves a verification F1 score of 0.9871.

[0107] Protection points of this embodiment:

[0108] 1. The RoBERTa Classifier composed of a pre-trained RoBERTa-base model and a linear classifier is used for log anomaly detection based on the low-rank subspace representation fine-tuning method.

[0109] 2. Introduce category weights and cross-entropy loss function with category weights to deal with the log category imbalance problem.

[0110] It should be noted that the ESN system: ESN (Educational Security Number) is the abbreviation of the Education Trusted Identity Application Verification Management Service Center system of the Ministry of Education. Its main function is to generate, issue and query unique and secure identity codes for personnel in the education industry (including students and teachers, etc.).

[0111] GELU: The GELU (Gaussian Error Linear Unit) activation function was proposed by Dan Hendrycks and Kevin Gimpel in 2016. It is a nonlinear activation function based on the Gaussian distribution. GELU can be considered a compromise between the ReLU and Sigmoid / Tanh activation functions, combining the non-saturation properties of ReLU with the smoothness properties of Sigmoid / Tanh.

[0112] RoBERTa: The RoBERTa model is an improved version of BERT (A Robustly Optimized BERT). Building on BERT's language masking strategy, it modifies key hyperparameters in BERT, including removing BERT's next-sentence pre-training objective and using a larger batch size and learning rate for training. RoBERTa also receives an order of magnitude more training than BERT, taking longer. This enables RoBERTa to generalize better to downstream tasks than BERT. RoBERTa-base is the base RoBERTa model using a 12-layer Transformer encoder.

[0113] This embodiment obtains raw log data and converts it into a target log sequence through log preprocessing. A preset base model is obtained and fine-tuned using low-rank linear subspace representation fine-tuning to generate a target model. The target log sequence is input into the target model to determine a feature vector. A preset linear classifier is then used to detect log anomalies based on the feature vector. By intervening in the representation of known encodings of specific semantic information, the low-rank linear subspace representation fine-tuning method can provide insights into how the model processes and understands data, reducing model training costs and improving model interpretability.

[0114] In addition, an embodiment of the present application also proposes a computer-readable storage medium, on which is stored a program for ESN log anomaly detection based on low-rank linear subspace representation fine-tuning. When the program for ESN log anomaly detection based on low-rank linear subspace representation fine-tuning is executed by a processor, the steps of the method for ESN log anomaly detection based on low-rank linear subspace representation fine-tuning as described above are implemented.

[0115] Reference Figure 6, Figure 6 This is a structural block diagram of an embodiment of the ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning of the present application.

[0116] like Figure 6 As shown, the ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning proposed in the embodiment of the present application includes:

[0117] The data acquisition module 10 is used to acquire original log data and convert the original log data into a target log sequence through log preprocessing;

[0118] A model generation module 20 is used to obtain a preset base model and adjust the preset base model through a low-rank linear subspace representation fine-tuning technique to generate a target model;

[0119] a feature vector module 30 for inputting a target log sequence into a target model to determine a feature vector;

[0120] The anomaly detection module 40 is configured to perform log anomaly detection based on the feature vector using a preset linear classifier.

[0121] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any restrictions on this.

[0122] This embodiment obtains raw log data and converts it into a target log sequence through log preprocessing. A preset base model is obtained and fine-tuned using low-rank linear subspace representation fine-tuning to generate a target model. The target log sequence is input into the target model to determine a feature vector. A preset linear classifier is then used to detect log anomalies based on the feature vector. By intervening in the representation of known encodings of specific semantic information, the low-rank linear subspace representation fine-tuning method can provide insights into how the model processes and understands data, reducing model training costs and improving model interpretability.

[0123] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In actual applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of this embodiment scheme, and no restrictions are imposed here.

[0124] In addition, for technical details not fully described in this embodiment, please refer to the method for ESN log anomaly detection based on low-rank linear subspace representation fine-tuning provided in any embodiment of the present application, which will not be repeated here.

[0125] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0126] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0127] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0128] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting anomalies in ESN logs based on fine-tuning of low-rank linear subspace representation, characterized in that: include: Obtaining original log data and converting the original log data into a target log sequence through log preprocessing; Obtaining a preset base model, and adjusting the preset base model through a low-rank linear subspace representation fine-tuning technique to generate a target model; inputting the target log sequence into the target model to determine a feature vector; Performing log anomaly detection based on the feature vector using a preset linear classifier; Wherein, the preset base model is the RoBERTa-base model; The step of adjusting the preset base model by using the low-rank linear subspace representation fine-tuning technology to generate a target model includes: Use the learned linear projection to adjust the representation of the preset base model in the subspace, ： in To learn the parameters, the original parameters of the preset base model remain unchanged. is the hidden representation of the applied intervention layer, is a low-rank matrix with orthogonal rows, and are the linear projection matrix and bias vector respectively; The low-rank linear subspace representation fine-tuning technique is applied to the first two and last two bits of the output representation of the 12th attention layer of the base model RoBERTa-base to generate a target model with LoReFT intervention.

2. The ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning according to claim 1 is characterized in that: The steps of obtaining original log data and converting the original log data into a target log sequence by log preprocessing include: Get raw log data; Parsing the raw log data into a set of log templates using the log parser Drain, and assigning an event ID to each log template in the set of log templates; Performing conversion according to the log template set to generate a structured log set; The target log sequence is generated by cutting the log set in a sliding window or session manner.

3. The ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning according to claim 1 is characterized in that: The step of inputting the target log sequence into the target model to determine a feature vector comprises: The target log sequence Input into the target model so that after the data is processed by Embedding and L-layer Transformer Enconder, the feature vectors of all log events in the log sequence are obtained. ; in After Embedding .

4. The ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning according to claim 1 is characterized in that: After the step of obtaining a preset base model and adjusting the preset base model by a low-rank linear subspace representation fine-tuning technique to generate a target model, the method further includes: A class weight mechanism is introduced, and the target model is trained using a cross-entropy loss function with class weights; No. Class weights of classes The calculation is: in, is the total number of samples in the dataset, is the total number of categories, It is The number of samples in the class; The cross entropy loss function with class weights is: in is the total number of categories, It is The weight of the class, It is The true label of the class, The model predicts The probability of the class.

5. The ESN log anomaly detection method based on low-rank linear subspace representation fine-tuning according to claim 1 is characterized in that: Before the step of performing log anomaly detection according to the feature vector using a preset linear classifier, the method further includes: Constructing a preset linear classifier, using a hyperbolic tangent function to approximate a standard GELU as an activation function of the preset linear classifier; in, is the input vector.

6. An ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning, characterized by: The method according to claim 1 is performed, wherein the ESN log anomaly detection system based on low-rank linear subspace representation fine-tuning comprises: A data acquisition module is used to acquire original log data and convert the original log data into a target log sequence through log preprocessing; A model generation module is used to obtain a preset base model and adjust the preset base model through a low-rank linear subspace representation fine-tuning technique to generate a target model; a feature vector module, configured to input the target log sequence into the target model to determine a feature vector; The anomaly detection module is used to perform log anomaly detection based on the feature vector using a preset linear classifier.

7. A computer device, characterized in that: The device comprises: a memory and a processor, wherein the processor executes the method according to any one of claims 1 to 5 when running computer instructions stored in the memory.

8. A computer-readable storage medium, characterized in that The method comprises instructions which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 5.

Images