Log anomaly detection method, device, equipment, storage medium and program product

Abstract

The present invention discloses a log anomaly detection method, device, equipment, storage medium and program product. The method comprises: extracting log semantic information, timestamp information and log key information from structured log data, determining the log time interval based on the timestamp information, generating time coding data according to the log semantic information and the log time interval, performing feature enhancement on the time coding data to obtain time feature information, generating log key coding data based on the log key information, performing feature enhancement on the log key coding data to obtain log key feature information, performing feature fusion on the time feature information and the log key feature information to obtain fused feature information, performing log anomaly detection based on the fused feature information after feature weight adjustment processing, and performing analysis in combination with the time feature and structure feature of the log, thereby effectively capturing the global information and local information of the log, and greatly improving the efficiency and accuracy of log anomaly detection.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a log anomaly detection method, device, equipment, storage medium and program product. Background Art

[0002] Log anomaly detection is an important research direction in the field of information technology. It involves automatically identifying and classifying abnormal behaviors from system logs in order to promptly detect and respond to potential security threats or system failures. As the number and scope of network information systems continue to increase, cyberspace faces growing security vulnerabilities. These risks include not only typical data leaks, but also increasingly complex and covert network attacks.

[0003] At present, log anomaly detection technology mainly relies on machine learning and deep learning methods. Machine learning methods identify normal and abnormal log patterns by training models. Although log anomaly detection technology has made certain progress, the existing log anomaly detection still faces problems such as data instability, data imbalance, anomaly diversity, label availability, real-time requirements, big data processing, log parsing and feature extraction, model interpretability, the emergence of unknown logs, and data enhancement methods. At present, the log anomaly detection technology based on deep learning needs to rely on a large amount of normal log data and a large amount of computing resources. In actual application, the logs that need to be detected are usually long text logs with large data volumes. Due to the limitations of computing resources and resource costs, it is difficult to fully capture the correlation features between log events, resulting in low efficiency and accuracy of log anomaly detection. Summary of the invention

[0004] The main purpose of the present invention is to provide a log anomaly detection method, device, equipment, storage medium and program product, aiming to solve the technical problem that the prior art cannot effectively capture the global information and local information of the log, resulting in low efficiency and accuracy of log anomaly detection.

[0005] To achieve the above object, the present invention provides a log anomaly detection method, which comprises the following steps:

[0006] Preprocess the collected raw log data to obtain structured log data;

[0007] Extracting log semantic information, timestamp information and log key information from the structured log data, and determining a log time interval based on the timestamp information;

[0008] Generate time coding data according to the log semantic information and the log time interval, and perform feature enhancement processing on the time coding data to obtain time feature information;

[0009] Generating log key encoding data based on the log key information, and performing feature enhancement processing on the log key encoding data to obtain log key feature information;

[0010] Fusing the time feature information with the log key feature information to obtain fused feature information;

[0011] The fused feature information is subjected to feature weight adjustment processing, and log anomaly detection is performed based on the fused feature information after the feature weight adjustment processing.

[0012] Optionally, the preprocessing of the collected raw log data to obtain structured log data includes:

[0013] Preprocessing the collected raw log data to obtain raw log sequence data, wherein the preprocessing includes data cleaning, data standardization, and missing data filling;

[0014] Generate an initial positioning log key, and embed the initial positioning log key into the original log sequence to obtain a candidate log sequence;

[0015] Generate a log key embedding matrix, and perform position encoding based on the log key embedding matrix to obtain the log key embedding position. The position encoding refers to the following formula:

[0016]

[0017]

[0018] in, Indicates the embedding position, represents the position index, which is the embedded position of the log key. represents an even dimension, represents odd dimensions, represents the length of the vector, represents the sine function, represents the cosine function;

[0019] More than one log key is embedded into the candidate log sequence based on the log key embedding position to obtain structured log data.

[0020] Optionally, generating time coding data according to the log semantic information and the log time interval, and performing feature enhancement processing on the time coding data to obtain time feature information includes:

[0021] Encode the log time interval through the time encoder to obtain initial time encoding data:

[0022]

[0023] in, It's time No. The initial time encoding data of the moment vector of dimensions, is a periodic activation function, wherein the periodic activation function includes a sine function, and represents the learnable encoding parameters;

[0024] Encoding the log semantic information through a text encoder to obtain text encoding data;

[0025] The initial time coding data and the text coding data are coded and merged to obtain time coding data;

[0026] Comparing the performance of a teacher model and a student model in a pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result, the Transformer encoder comprising a plurality of Transformer layers, the Transformer layers comprising a multi-head self-attention relationship distillation module, the multi-head self-attention relationship distillation module comprising the teacher model and the student model;

[0027] The time-coded data is input into the optimized Transformer encoder for feature enhancement processing to obtain time feature information.

[0028] Optionally, the comparing the performance of the teacher model and the student model in the pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result, includes:

[0029] Determine the weighted sum parameters of the self-attention relationship loss between the teacher model and the student model in the Transformer encoder:

[0030]

[0031] in, It is and The relative entropy parameter between the self-attention relations, The calculation formula is as follows:

[0032]

[0033] in, and Represents the teacher model Layer and Student Model The self-attention relationship of the layer, represents the index of the attention head, is the number of relation heads, is the length of the sequence, is the weight assigned to each self-attention relation loss, represents the time step, and the relative entropy parameter is used to measure the distribution difference between the teacher model and the student model in the self-attention relationship;

[0034] The teacher model and the student model are compared in performance based on the weighted and parameter, and the Transformer encoder is optimized according to the performance comparison result.

[0035] Optionally, generating log key encoding data based on the log key information, and performing feature enhancement processing on the log key encoding data to obtain log key feature information includes:

[0036] Generate log key encoding data based on the log key information;

[0037] Performing average pooling and maximum pooling on the log key encoded data to generate a first spatial context descriptor and a second spatial context descriptor;

[0038] Generate a channel attention feature based on the first spatial context descriptor and the second spatial context descriptor:

[0039]

[0040] in, represents the channel attention feature, is the sigmoid function, MLP stands for multi-layer perceptron, and are the weights of the MLP, and Both represent the first spatial context descriptor, and Both represent the second spatial context descriptor;

[0041] Performing average pooling and maximum pooling on the log key encoded data in a channel dimension to generate a first feature map and a second feature map;

[0042] Generate a spatial attention feature based on the first feature map and the second feature map:

[0043]

[0044] in, represents the spatial attention feature, represents the convolution operation, is the sigmoid function, and Both represent the first feature map, and Both represent the second feature map;

[0045] The log key encoded data is subjected to feature enhancement processing according to the channel attention feature and the spatial attention feature to obtain log key feature information:

[0046]

[0047] in, Indicates log key feature information. Represents log key-encoded data, represents the channel attention feature, Represents spatial attention features.

[0048] Optionally, the performing feature weight adjustment processing on the fused feature information, and performing log anomaly detection based on the fused feature information after the feature weight adjustment processing, includes:

[0049] Perform linear transformation on the fused feature information to generate a query vector, a key vector, and a value vector:

[0050]

[0051] in, represents the query vector, represents the key vector, represents a value vector, represents the fusion feature information, represents the weight matrix for generating query vectors, represents the weight matrix for generating the key vector, The weight matrix representing the generated value vector;

[0052] The query vector, the key vector and the value vector are weighted averaged, and the fused feature information is subjected to feature weight adjustment processing based on the weighted average result to obtain the fused feature information after the feature weight adjustment processing:

[0053]

[0054] in, represents the dot product of vectors, represents the index of the feature dimension, It is dimensional attention vector, Indicates position offset, Represents a value vector In Location The row vector of It represents the fusion feature information after feature weight adjustment. It is used to express conditions or constraints. Represents the query vector In Location No. The elements of the feature dimension, Represents the key vector No. feature dimensions, Represents the key vector In Location No. The elements of the feature dimension, Represents the position deviation matrix Middle position With location Position deviation between

[0055] Log anomaly detection is performed based on the fused feature information after the feature weight adjustment process.

[0056] In addition, to achieve the above object, the present invention also provides a log anomaly detection device, the log anomaly detection device comprising:

[0057] The data processing module is used to pre-process the collected raw log data to obtain structured log data;

[0058] A data parsing module, used to extract log semantic information, timestamp information and log key information from the structured log data, and determine the log time interval based on the timestamp information;

[0059] A time feature processing module, used to generate time coding data according to the log semantic information and the log time interval, and perform feature enhancement processing on the time coding data to obtain time feature information;

[0060] A log key feature processing module, used to generate log key encoding data based on the log key information, and perform feature enhancement processing on the log key encoding data to obtain log key feature information;

[0061] A feature fusion module, used to fuse the time feature information with the log key feature information to obtain fused feature information;

[0062] The log anomaly detection module is used to perform feature weight adjustment processing on the fused feature information, and perform log anomaly detection based on the fused feature information after the feature weight adjustment processing.

[0063] In addition, to achieve the above-mentioned purpose, the present application also proposes a log anomaly detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the log anomaly detection method as described above.

[0064] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the log anomaly detection method described above are implemented.

[0065] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the log anomaly detection method described above are implemented.

[0066] The present invention obtains structured log data by preprocessing the collected original log data, extracts log semantic information, timestamp information and log key information from the structured log data, determines the log time interval based on the timestamp information, generates time coding data according to the log semantic information and the log time interval, performs feature enhancement processing on the time coding data to obtain time feature information, generates log key coding data based on the log key information, performs feature enhancement processing on the log key coding data to obtain log key feature information, performs feature fusion on the time feature information and the log key feature information to obtain fused feature information, performs feature weight adjustment processing on the fused feature information, and performs log anomaly detection based on the fused feature information after the feature weight adjustment processing; because the present invention encodes the time information and log key information of the log and performs feature enhancement processing, thereby combining the time feature and structural feature of the log for feature analysis, accurately captures the global information and local information of the log, effectively reduces the resource requirements and calculation time for processing long text logs, and greatly improves the efficiency and accuracy of log anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0069] Figure 1It is a structural diagram of a log anomaly detection device in a hardware operating environment involved in an embodiment of the present invention;

[0070] Figure 2 This is a flow chart of the first embodiment of the log anomaly detection method of the present invention;

[0071] Figure 3 This is a flow chart of a second embodiment of a log anomaly detection method according to the present invention;

[0072] Figure 4 Schematic diagram of the structure of a log anomaly detection model in an embodiment of a log anomaly detection method of the present invention;

[0073] Figure 5 A schematic diagram of a flow chart of a log anomaly detection model in an embodiment of a log anomaly detection method of the present invention performing log anomaly detection;

[0074] Figure 6 This is a structural block diagram of the first embodiment of the log anomaly detection device of the present invention.

[0075] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0076] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0077] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of the structure of a log anomaly detection device in a hardware operating environment according to an embodiment of the present invention.

[0078] like Figure 1As shown, the log anomaly detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0079] Those skilled in the art will understand that Figure 1 The structure shown in does not constitute a limitation on the log anomaly detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0080] like Figure 1 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a log anomaly detection program.

[0081] exist Figure 1 In the log anomaly detection device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the log anomaly detection device of the present invention can be set in the log anomaly detection device, and the log anomaly detection device calls the log anomaly detection program stored in the memory 1005 through the processor 1001, and executes the log anomaly detection method provided by the embodiment of the present invention.

[0082] The embodiment of the present invention provides a log anomaly detection method, referring to Figure 2 , Figure 2 The figure is a flow chart of the first embodiment of the log anomaly detection method of the present invention.

[0083] In this embodiment, the log anomaly detection method includes the following steps:

[0084] Step S10: pre-process the collected original log data to obtain structured log data.

[0085] It should be understood that in log anomaly detection, log data can be processed and analyzed through the Transformer encoder, but the Transformer encoder usually requires a large amount of computing resources and training data, and the layer normalization module in the Transformer encoder may block the gradient flow, causing the top-level gradient to disappear, so it is impossible to effectively capture the local information of the log, resulting in low efficiency and accuracy of log anomaly detection. This embodiment encodes and enhances the time interval and log key of the log, and then fuses the time feature information with the log key feature information, thereby combining the time features and structural features of the log data for analysis, accurately capturing the global and local information of the log, effectively reducing the resource requirements and computing time for processing long text logs, and greatly improving the efficiency and accuracy of log anomaly detection.

[0086] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a terminal electronic device capable of realizing the above functions, etc. The following takes a log anomaly detection device (detection device) as an example to illustrate this embodiment and the following embodiments. The detection device may include a log anomaly detection model, and the log anomaly detection model is constructed based on a Transformer encoder. The log anomaly detection model includes a Transformer encoder, and the Transformer encoder includes multiple Transformer layers, and each Transformer layer includes a multi-head self-attention module and a CBAM module feedforward sublayer.

[0087] In some embodiments, preprocessing may include data cleaning, formatting, standardization, etc. The detection device may generate structured log data by inserting a log key into the preprocessed raw log data.

[0088] In some embodiments, the detection device may load raw log data from a data source, and use an online log parsing algorithm Drain based on a fixed-depth tree to process the raw log data to obtain structured log data.

[0089] Furthermore, in order to improve the detection efficiency, the above step S10 may include:

[0090] Step S101: pre-processing the collected original log data to obtain original log sequence data;

[0091] Step S102: Generate an initial positioning log key, and embed the initial positioning log key into the original log sequence to obtain a candidate log sequence;

[0092] Step S103: Generate a log key embedding matrix, and perform position encoding based on the log key embedding matrix to obtain the log key embedding position;

[0093] Step S104: embedding one or more log keys into the candidate log sequence based on the log key embedding position to obtain structured log data.

[0094] It should be noted that the preprocessing includes data cleaning, data standardization and missing data filling.

[0095] In some embodiments, for a normal log sequence , the detection device can first add a special marker (DIST) at the beginning of the sequence as the first log key (i.e., the initial positioning log key) through the log anomaly detection model. Each log key is represented as the input representation , which is the sum of the log key embedding and the position embedding.

[0096] Log-key embedding matrix Randomly generated, the position embedding T is generated by a sine function to encode the position information of the log key in the sequence. The position information encoding refers to the following formula:

[0097]

[0098]

[0099] in, Indicates the embedding position, represents the position index, which is the embedded position of the log key. represents an even dimension, represents odd dimensions, represents the length of the vector, represents the sine function, Represents the cosine function.

[0100] Sine and cosine functions are used to generate position encodings because they are periodic and can effectively represent position information. By dividing the position index by a power of 10,000, it is ensured that the encodings of different positions are distinguished in different dimensions, thereby capturing the position information. As a denominator, it ensures that the scale of the encoding is not too large, thus avoiding the problem of numerical instability.

[0101] Step S20: extracting log semantic information, timestamp information and log key information from the structured log data, and determining a log time interval based on the timestamp information.

[0102] In some embodiments, the detection device may extract timestamp information from the structured log, and use a time difference algorithm to evaluate the time characteristics of the log event, thereby obtaining the event interval between the log events.

[0103] In some embodiments, the detection device can find the field containing time information according to the log format document or through regular expression matching, and obtain the time zone information and time format of the structured log data, extract the timestamp information based on the field containing time information, the time zone information and the time format, sort the log events based on the timestamp information, traverse the sorted log list, calculate the time difference between each adjacent log event, and thus obtain the log time interval.

[0104] In some embodiments, the detection device may obtain log semantic information through keyword extraction (eg, TD-IDF, etc.), rule matching, and natural language processing models.

[0105] Step S30: Generate time coding data according to the log semantic information and the log time interval, and perform feature enhancement processing on the time coding data to obtain time feature information.

[0106] In some embodiments, the detection device may use a trainable Time2Vec encoding and learnable parameters to generate a vector representation of the time interval, combining the time encoding vector with the semantic embedding vector of the log message.

[0107] Furthermore, in order to accurately capture the time characteristic information of the log, the above step S30 may include:

[0108] Step S301: Encode the log time interval by a time encoder to obtain initial time encoding data;

[0109] Step S302: Encode the log semantic information through a text encoder to obtain text encoding data;

[0110] Step S303: Encode and fuse the initial time code data with the text code data to obtain time code data;

[0111] Step S304: performing a performance comparison between a teacher model and a student model in a pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result;

[0112] Step S305: Input the time-coded data into the optimized Transformer encoder for feature enhancement processing to obtain time feature information.

[0113] It should be noted that the time encoder can be a Time2Vec encoder. The detection device can use the time interval feature Time2Vec encoding of the comprehensive time interval encoding information through the log anomaly detection model to generate a vector representation of the time interval. It uses learnable parameters to generate a vector representation of the time interval. The calculation formula of Time2Vec encoding is:

[0114]

[0115] in, It's time No. The initial time encoding data of the moment vector of dimensions (i.e. Time2Vec encoding), is a periodic activation function, wherein the periodic activation function includes a sine function, and Representing learnable encoding parameters, the detection device can use the time relative to the first event as the input of the encoding function.

[0116] It should be noted that the above-mentioned log anomaly detection model may include a Transformer encoder, the Transformer encoder is composed of multiple Transformer layers, the Transformer layer includes a multi-head self-attention relationship distillation module, and the multi-head self-attention relationship distillation module includes the teacher model and the student model.

[0117] Furthermore, in order to effectively optimize the encoding performance and improve the accuracy of anomaly detection, the above step S304 may include:

[0118] Step S3041: determining a weighted sum parameter of the self-attention relationship loss between the teacher model and the student model in the Transformer encoder;

[0119] Step S3042: performing a performance comparison between the teacher model and the student model based on the weighted sum parameters, and optimizing the Transformer encoder according to the performance comparison results.

[0120] It should be noted that, for the transformer encoder that introduces multi-head self-attention relation distillation, this embodiment evaluates the effect and accuracy of the student model on the anomaly detection task by comparing the performance of the teacher model and the student model. For the teacher model and the student model, even if they use different numbers of attention heads, knowledge distillation (KD) training can be performed by converting the query, key, and value vectors of different numbers of attention heads into the same number of relation head vectors.

[0121] It can be understood that this embodiment can use KL (Kullback-Leibler Divergence) divergence as the loss function of the self-attention relationship between the teacher model and the student model. The above KL divergence can be a relative entropy parameter. The KL divergence measures the distribution difference between the teacher model and the student model in the self-attention relationship.

[0122] In this embodiment, the total loss function It is defined as the weighted sum of all self-attention relation losses:

[0123]

[0124] in It is and The relative entropy parameter between the self-attention relationships (i.e., KL divergence), The calculation formula is as follows:

[0125]

[0126] in, and Represents the teacher model Layer and Student Model The self-attention relationship of the layer, represents the index of the attention head, is the number of relation heads, is the length of the sequence, is the weight assigned to each self-attention relation loss, represents the time step, and the relative entropy parameter is used to measure the distribution difference between the teacher model and the student model in the self-attention relationship.

[0127] Step S40: Generate log key encoding data based on the log key information, and perform feature enhancement processing on the log key encoding data to obtain log key feature information.

[0128] It should be noted that, in this embodiment, the CBAM module can be integrated in the Transformer encoder layer to process the log key encoded data and enhance the feature representation.

[0129] It should be noted that the Transformer encoder consists of multiple Transformer layers, each of which includes a multi-head self-attention and CBAM module feed-forward sublayer. Parallel self-attentions are used to jointly capture different aspects of information at different positions in the input log sequence. For the first head of the attention layer, the scaled dot product self-attention is defined as:

[0130]

[0131] in , is the input representation of the log sequence; It is The linear projection weight of each head, It is the dimension of a head.

[0132] Long attention will Connect the heads in parallel:

[0133]

[0134] in is the projection matrix, is the output dimension of the multi-head attention sub-layer.

[0135] Furthermore, in order to capture potential dependencies in the logs and improve feature expression capabilities, thereby accurately extracting the temporal and spatial information of the logs, the above step S40 may include:

[0136] Step S401: generating log key encoding data based on the log key information;

[0137] Step S402: performing average pooling processing and maximum pooling processing on the log key encoded data to generate a first spatial context descriptor and a second spatial context descriptor;

[0138] Step S403: generating a channel attention feature based on the first spatial context descriptor and the second spatial context descriptor;

[0139] Step S404: performing average pooling and maximum pooling processing on the log key encoded data in the channel dimension to generate a first feature map and a second feature map;

[0140] Step S405: generating a spatial attention feature based on the first feature map and the second feature map;

[0141] Step S406: Perform feature enhancement processing on the log key encoded data according to the channel attention feature and the spatial attention feature to obtain log key feature information.

[0142] It should be noted that the Transformer encoder in the log anomaly detection model may include a CBAM module, which includes a channel attention module and a spatial attention module. The channel attention module can generate a channel attention map by utilizing the channel relationship between features. The spatial attention module can generate a spatial attention map by utilizing the spatial relationship between features.

[0143] It should be understood that the detection device can calculate the attention features of the log key encoding data from the channel dimension and the spatial dimension respectively through the CBAM module in the log anomaly detection model to obtain the channel attention features and the spatial attention features. The output log key feature information of the CBAM module is referred to the following formula:

[0144]

[0145] in, Indicates log key feature information. Represents log key-encoded data, represents the channel attention feature, Represents spatial attention features.

[0146] It should be noted that in the feedforward sublayer of the Transformer of the log anomaly detection model, the input x (i.e., the log key encoded data) first passes through two linear layers ( and ), the ReLU activation function and Dropout regularization are applied to the hidden representation of each activity separately. Then, the output of the CBAM module is added to the output of the FFN to form a residual connection. The final output is:

[0147]

[0148] It should be noted that the detection device can aggregate spatial information through average pooling (Adaptive Avg Pool1d) and maximum pooling (Adaptive Max Pool 1d) on the log key encoding data to generate two different spatial context descriptors: and The two descriptors are processed through a shared multi-layer perceptron (MLP) to produce a channel attention map The calculation formula of channel attention feature is:

[0149]

[0150] in, represents the channel attention feature, is the sigmoid function, MLP stands for multi-layer perceptron, and are the weights of the MLP, and Both represent the first spatial context descriptor, and Both represent the second spatial context descriptor.

[0151] It should be noted that the detection device can generate two 2D graphs by applying average pooling and maximum pooling operations on the channel dimension to the log-key encoded data. and The two images are concatenated and processed through a convolutional layer (Conv1d) to produce a spatial attention map The calculation formula of spatial attention feature is:

[0152]

[0153] in, represents the spatial attention feature, represents the convolution operation, is the sigmoid function, and Both represent the first feature map, and Both represent the second feature map.

[0154] Step S50: Fusing the time feature information with the log key feature information to obtain fused feature information.

[0155] It should be noted that the log anomaly detection model of this embodiment also includes a feature data integration layer. The detection device fuses the time feature information with the log key feature information through the feature data integration layer in the log anomaly detection model to obtain a comprehensive feature representation (i.e., fused feature information).

[0156] In some embodiments, the detection device may dynamically determine the importance of the time feature and the log key feature according to the context, and use a self-attention mechanism to fuse the time feature information with the log key feature information, thereby better fusing the two types of features.

[0157] Step S60: performing feature weight adjustment processing on the fused feature information, and performing log anomaly detection based on the fused feature information after the feature weight adjustment processing.

[0158] It should be noted that the log anomaly detection module of this embodiment also includes an AFT (Additive Feature Tokenization) fully connected layer. This embodiment performs deeper feature fusion and processing by transferring the comprehensive feature representation to the AFT fully connected layer.

[0159] In some embodiments, the log anomaly detection model can be further processed by the self-supervised task layer based on the output and comprehensive feature representation of the AFT layer to generate the final anomaly detection result in the output layer.

[0160] In some embodiments, the detection device improves and optimizes the Transformer model to construct a log anomaly detection model, wherein the log anomaly detection model includes a Transformer encoder, wherein the Transformer encoder includes multiple Transformer layers, and each Transformer layer includes a multi-head self-attention and a feedforward sublayer combined with a CBAM module. The detection device can encode information by adding a time interval, using a log anomaly detection method that adds a CBAM feedforward layer and uses an AFT (attention free transformer) module on the comprehensive feature data. The focus is to reduce the computing time and memory requirements of the transformer model when processing long texts. By parameterizing the position bias, especially the re-parameterization of the one-dimensional convolution, the model is given stronger adaptability and expressiveness, and the model's understanding and utilization of long-distance context information is enhanced. It is particularly beneficial for applications that require real-time processing of large amounts of text. At the same time, the method optimizes resource allocation, dynamically adjusts the allocation of computing resources through the attention mechanism, and reduces the processing of irrelevant information, thereby possibly improving the efficiency and response speed of the model and reducing dependence on computing resources.

[0161] This embodiment obtains structured log data by preprocessing the collected original log data, extracts log semantic information, timestamp information and log key information from the structured log data, determines the log time interval based on the timestamp information, generates time coding data according to the log semantic information and the log time interval, performs feature enhancement processing on the time coding data to obtain time feature information, generates log key coding data based on the log key information, performs feature enhancement processing on the log key coding data to obtain log key feature information, performs feature fusion on the time feature information and the log key feature information to obtain fused feature information, performs feature weight adjustment processing on the fused feature information, and performs log anomaly detection based on the fused feature information after the feature weight adjustment processing; because this embodiment encodes the time information and log key information of the log and performs feature enhancement processing, thereby combining the time feature and structural feature of the log for feature analysis, accurately captures the global information and local information of the log, effectively reduces the resource requirements and computing time for processing long text logs, and greatly improves the efficiency and accuracy of log anomaly detection.

[0162] refer to Figure 3 , Figure 3 This is a flow chart of the second embodiment of the log anomaly detection method of the present invention.

[0163] Based on the above first embodiment, in this embodiment, step S60 further includes:

[0164] Step S601: performing linear transformation on the fused feature information to generate a query vector, a key vector and a value vector.

[0165] It should be noted that the detection device can first convert the input fusion feature information linearly into a query vector through the linear layer in the log anomaly detection model , key vector Sum value vector :

[0166]

[0167] in, represents the query vector, represents the key vector, represents a value vector, represents the fusion feature information, represents the weight matrix for generating query vectors, represents the weight matrix for generating the key vector, A weight matrix representing the generated value vector.

[0168] Step S602: performing weighted averaging on the query vector, the key vector and the value vector, and performing feature weight adjustment processing on the fused feature information based on the weighted averaging result to obtain fused feature information after feature weight adjustment processing.

[0169] It should be noted that for each target position , a weighted average of the values ​​is performed through an AFT fully connected layer, and the result is combined with the query through element-wise multiplication. The weights of the weighted average are determined by the key and a set of learned pairwise position biases w The operation of the AFT fully connected layer can be expressed as:

[0170]

[0171] in, represents the dot product of vectors, represents the index of the feature dimension, It is dimensional attention vector, Indicates position offset, Represents a value vector In Location The row vector of It represents the fusion feature information after feature weight adjustment. It is used to express conditions or constraints. Represents the query vector In Location No. The elements of the feature dimension, Represents the key vector No. feature dimensions, Represents the key vector In Location No. The elements of the feature dimension, Represents the position deviation matrix Middle position With location The position deviation between them.

[0172] Step S603: performing log anomaly detection based on the fused feature information after the feature weight adjustment process.

[0173] In some embodiments, the operation of the AFT fully connected layer can also be expressed as:

[0174]

[0175]

[0176] in, Represents the output of the AFT layer, the dimension is . Represents the entire AFT layer for the input sequence The output sequence obtained after the operation. Represents input exist The element is a dimensional vector. and is the position index in the sequence, Indicates the target location, Indicates the traversal position. Indicates the length of the sequence. represents element-wise multiplication, is the non-linear activation function applied to the query, defaults to sigmoid. Represents a query ( ) at position The row vector of . Represents value ( ) at position The row vector of . Indicates the key ( ) at position The row vector of .

[0177] In some embodiments, the detection device can perform anomaly detection on the log by building a log anomaly detection model. Figure 4 , Figure 4 It is a structural diagram of a log anomaly detection model, wherein the log anomaly detection model includes a Transformer encoder, wherein the Transformer encoder includes an input layer, multiple Transformer layers, a feature data synthesis layer, an AFT fully connected layer, a self-supervised task layer, and an output layer, wherein each Transformer layer includes a multi-head self-attention module and a CBAM module feedforward sublayer, wherein the Transformer layer is used to perform feature processing on time-coded data and log key-coded data; the feature data synthesis layer is used to perform feature fusion on time feature information and log key feature information to obtain fused feature information; the AFT fully connected layer is used to perform deeper feature fusion and processing on fused feature information; the self-supervised task layer is used to train a model based on fused feature information and fused feature information output by the AFT fully connected layer, and enhance the generalization ability of the model; Figure 4 middle is the time characteristic information, Log key feature information.

[0178] Reference Figure 5 , Figure 5The flow chart of log anomaly detection model is as follows: the log anomaly detection model obtains the original log information and pre-processes the collected log data, including cleaning, formatting, extracting the log key information and time interval encoding information of the log information; adding the CBAM module to the Feedforward layer of the Transformer encoder to improve the long-distance dependency modeling capability with strong feature representation, and more accurately capture and identify abnormal patterns; using the all-MiniLM-L6-v2 model to process the time interval encoded log data. The CBAM-improved Transformer encoder is used to process the log key encoding data; the processed data is added to obtain the comprehensive feature data; the AFT module is used to process the added comprehensive feature data. Anomaly detection is performed based on the results of the AFT module processing.

[0179] This embodiment performs linear transformation on the fused feature information to generate a query vector, a key vector and a value vector, performs weighted average on the query vector, the key vector and the value vector, and performs feature weight adjustment processing on the fused feature information based on the weighted average result to obtain the fused feature information after the feature weight adjustment processing, and performs log anomaly detection based on the fused feature information after the feature weight adjustment processing to achieve deeper feature fusion and processing, thereby accurately mining the potential risk features in the log data and greatly improving the accuracy of log anomaly detection.

[0180] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a log anomaly detection program is stored. When the log anomaly detection program is executed by a processor, the steps of the log anomaly detection method described above are implemented.

[0181] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0182] The computer-readable storage medium may be included in the log anomaly detection device; or may exist independently without being assembled into the log anomaly detection device.

[0183] In addition, an embodiment of the present invention further provides a computer program product, including a log anomaly detection program, which implements the steps of the log anomaly detection method described above when executed by a processor.

[0184] The specific implementation manner of the computer program product of the present invention is basically the same as the above-mentioned embodiments of the log anomaly detection method, and will not be repeated here.

[0185] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the log anomaly detection device of the present invention.

[0186] like Figure 6 As shown, the log anomaly detection device proposed in the embodiment of the present invention includes:

[0187] The data processing module 10 is used to pre-process the collected raw log data to obtain structured log data;

[0188] A data parsing module 20, for extracting log semantic information, timestamp information and log key information from the structured log data, and determining a log time interval based on the timestamp information;

[0189] A time feature processing module 30, configured to generate time coding data according to the log semantic information and the log time interval, and perform feature enhancement processing on the time coding data to obtain time feature information;

[0190] A log key feature processing module 40 is used to generate log key encoding data based on the log key information, and perform feature enhancement processing on the log key encoding data to obtain log key feature information;

[0191] A feature fusion module 50 is used to fuse the time feature information with the log key feature information to obtain fused feature information;

[0192] The log anomaly detection module 60 is used to perform feature weight adjustment processing on the fused feature information, and perform log anomaly detection based on the fused feature information after the feature weight adjustment processing.

[0193] Furthermore, the data processing module 10 is also used to pre-process the collected original log data to obtain original log sequence data, wherein the pre-processing includes data cleaning, data standardization and missing data filling;

[0194] Generate an initial positioning log key, and embed the initial positioning log key into the original log sequence to obtain a candidate log sequence;

[0195] Generate a log key embedding matrix, and perform position encoding based on the log key embedding matrix to obtain the log key embedding position. The position encoding refers to the following formula:

[0196]

[0197]

[0198] in, Indicates the embedding position, represents the position index, which is the embedded position of the log key. represents an even dimension, represents odd dimensions, represents the length of the vector, represents the sine function, represents the cosine function;

[0199] More than one log key is embedded into the candidate log sequence based on the log key embedding position to obtain structured log data.

[0200] Furthermore, the time feature processing module 30 is further used to encode the log time interval through a time encoder to obtain initial time encoding data:

[0201]

[0202] in, It's time No. The initial time encoding data of the moment vector of dimensions, is a periodic activation function, wherein the periodic activation function includes a sine function, and represents the learnable encoding parameters;

[0203] Encoding the log semantic information through a text encoder to obtain text encoding data;

[0204] The initial time coding data and the text coding data are coded and merged to obtain time coding data;

[0205] Comparing the performance of a teacher model and a student model in a pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result, the Transformer encoder comprising a plurality of Transformer layers, the Transformer layers comprising a multi-head self-attention relationship distillation module, the multi-head self-attention relationship distillation module comprising the teacher model and the student model;

[0206] The time-coded data is input into the optimized Transformer encoder for feature enhancement processing to obtain time feature information.

[0207] Furthermore, the temporal feature processing module 30 is further used to determine a weighted sum parameter of the self-attention relationship loss between the teacher model and the student model in the Transformer encoder:

[0208]

[0209] Among them, the relative entropy parameter of The calculation formula is as follows:

[0210]

[0211] in, and Represents the teacher model Layer and Student Model The self-attention relationship of the layer, represents the index of the attention head, is the number of relation heads, is the length of the sequence, is the weight assigned to each self-attention relation loss, represents the time step, and the relative entropy parameter is used to measure the distribution difference between the teacher model and the student model in the self-attention relationship;

[0212] The teacher model and the student model are compared in performance based on the weighted and parameter, and the Transformer encoder is optimized according to the performance comparison result.

[0213] Furthermore, the log key feature processing module 40 is also used to generate log key encoding data based on the log key information;

[0214] Performing average pooling and maximum pooling on the log key encoded data to generate a first spatial context descriptor and a second spatial context descriptor;

[0215] Generate a channel attention feature based on the first spatial context descriptor and the second spatial context descriptor:

[0216]

[0217] in, represents the channel attention feature, is the sigmoid function, MLP stands for multi-layer perceptron, and are the weights of the MLP, and Both represent the first spatial context descriptor, and Both represent the second spatial context descriptor;

[0218] Performing average pooling and maximum pooling on the log key encoded data in a channel dimension to generate a first feature map and a second feature map;

[0219] Generate a spatial attention feature based on the first feature map and the second feature map:

[0220]

[0221] in, represents the spatial attention feature, represents the convolution operation, is the sigmoid function, and Both represent the first feature map, and Both represent the second feature map;

[0222] The log key encoded data is subjected to feature enhancement processing according to the channel attention feature and the spatial attention feature to obtain log key feature information:

[0223]

[0224] in, Indicates log key feature information. Represents log key-encoded data, represents the channel attention feature, Represents spatial attention features.

[0225] Furthermore, the log anomaly detection module 60 is further used to perform linear transformation on the fused feature information to generate a query vector, a key vector and a value vector:

[0226]

[0227] in, represents the query vector, represents the key vector, represents a value vector, represents the fusion feature information, represents the weight matrix for generating query vectors, represents the weight matrix for generating the key vector, The weight matrix representing the generated value vector;

[0228] The query vector, the key vector and the value vector are weighted averaged, and the fused feature information is subjected to feature weight adjustment processing based on the weighted average result to obtain the fused feature information after the feature weight adjustment processing:

[0229]

[0230] in, represents the dot product of vectors, represents the index of the feature dimension, It is dimensional attention vector, Indicates position offset, Represents a value vector In Location The row vector of It represents the fusion feature information after feature weight adjustment. It is used to express conditions or constraints. Represents the query vector In Location No. The elements of the feature dimension, Represents the key vector No. feature dimensions, Represents the key vector In Location No. The elements of the feature dimension, Represents the position deviation matrix Middle position With location Position deviation between

[0231] Log anomaly detection is performed based on the fused feature information after the feature weight adjustment process.

[0232] This embodiment obtains structured log data by preprocessing the collected original log data, extracts log semantic information, timestamp information and log key information from the structured log data, determines the log time interval based on the timestamp information, generates time coding data according to the log semantic information and the log time interval, performs feature enhancement processing on the time coding data to obtain time feature information, generates log key coding data based on the log key information, performs feature enhancement processing on the log key coding data to obtain log key feature information, performs feature fusion on the time feature information and the log key feature information to obtain fused feature information, performs feature weight adjustment processing on the fused feature information, and performs log anomaly detection based on the fused feature information after the feature weight adjustment processing; because this embodiment encodes the time feature information and the log key feature information of the log and performs feature enhancement processing, thereby combining the time feature and the structural feature of the log for feature analysis, accurately captures the global information and local information of the log, effectively reduces the resource requirements and computing time for processing long text logs, and greatly improves the efficiency and accuracy of log anomaly detection.

[0233] The log anomaly detection device provided by the present application adopts the log anomaly detection method in the above embodiment, which can solve the technical problem of log anomaly detection. Compared with the prior art, the beneficial effects of the log anomaly detection device provided by the present application are the same as the beneficial effects of the log anomaly detection method provided by the above embodiment, and the other technical features in the log anomaly detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0234] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0235] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0236] In addition, for technical details not fully described in this embodiment, reference can be made to the log anomaly detection method provided in any embodiment of the present invention, and will not be repeated here.

[0237] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0238] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0239] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0240] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A log anomaly detection method, characterized in that: The log anomaly detection method comprises: Preprocess the collected raw log data to obtain structured log data; Extracting log semantic information, timestamp information and log key information from the structured log data, and determining a log time interval based on the timestamp information; Generate time coding data according to the log semantic information and the log time interval, and perform feature enhancement processing on the time coding data to obtain time feature information; Generating log key encoding data based on the log key information, and performing feature enhancement processing on the log key encoding data to obtain log key feature information; Fusing the time feature information with the log key feature information to obtain fused feature information; Performing feature weight adjustment processing on the fused feature information, and performing log anomaly detection based on the fused feature information after the feature weight adjustment processing; The step of generating time coding data according to the log semantic information and the log time interval, and performing feature enhancement processing on the time coding data to obtain time feature information includes: Encode the log time interval through the time encoder to obtain initial time encoding data: in, It's time No. The initial time encoding data of the moment vector of dimensions, is a periodic activation function, wherein the periodic activation function includes a sine function, and represents the learnable encoding parameters; Encoding the log semantic information through a text encoder to obtain text encoding data; The initial time coding data and the text coding data are coded and merged to obtain time coding data; Comparing the performance of a teacher model and a student model in a pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result, the Transformer encoder comprising a plurality of Transformer layers, the Transformer layers comprising a multi-head self-attention relationship distillation module, the multi-head self-attention relationship distillation module comprising the teacher model and the student model; The time-coded data is input into the optimized Transformer encoder for feature enhancement processing to obtain time feature information.

2. The log anomaly detection method according to claim 1, characterized in that: The preprocessing of the collected raw log data to obtain structured log data includes: Preprocessing the collected raw log data to obtain raw log sequence data, wherein the preprocessing includes data cleaning, data standardization, and missing data filling; Generate an initial positioning log key, and embed the initial positioning log key into the original log sequence to obtain a candidate log sequence; Generate a log key embedding matrix, and perform position encoding based on the log key embedding matrix to obtain the log key embedding position. The position encoding refers to the following formula: in, Indicates the embedding position, represents the position index, which is the embedded position of the log key. represents an even dimension, represents odd dimensions, represents the length of the vector, represents the sine function, represents the cosine function; More than one log key is embedded into the candidate log sequence based on the log key embedding position to obtain structured log data.

3. The log anomaly detection method according to claim 2, characterized in that: The comparing the performance of the teacher model and the student model in the pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result, includes: Determine the weighted sum parameters of the self-attention relationship loss between the teacher model and the student model in the Transformer encoder: in, It is and The relative entropy parameter between the self-attention relations, The calculation formula is as follows: in, and Represents the teacher model Layer and Student Model The self-attention relationship of the layer, represents the index of the attention head, is the number of relation heads, is the length of the sequence, is the weight assigned to each self-attention relation loss, represents the time step, and the relative entropy parameter is used to measure the distribution difference between the teacher model and the student model in the self-attention relationship; The teacher model and the student model are compared in performance based on the weighted and parameter, and the Transformer encoder is optimized according to the performance comparison result.

4. The log anomaly detection method according to claim 3, characterized in that: The generating log key encoding data based on the log key information, and performing feature enhancement processing on the log key encoding data to obtain log key feature information includes: Generate log key encoding data based on the log key information; Performing average pooling and maximum pooling on the log key encoded data to generate a first spatial context descriptor and a second spatial context descriptor; Generate a channel attention feature based on the first spatial context descriptor and the second spatial context descriptor: in, represents the channel attention feature, is the sigmoid function, MLP stands for multi-layer perceptron, and are the weights of the MLP, and Both represent the first spatial context descriptor, and Both represent the second spatial context descriptor; Performing average pooling and maximum pooling on the log key encoded data in a channel dimension to generate a first feature map and a second feature map; Generate a spatial attention feature based on the first feature map and the second feature map: in, represents the spatial attention feature, represents the convolution operation, is the sigmoid function, and Both represent the first feature map, and Both represent the second feature map; The log key encoded data is subjected to feature enhancement processing according to the channel attention feature and the spatial attention feature to obtain log key feature information: in, Indicates log key feature information. Represents log key-encoded data, represents the channel attention feature, Represents spatial attention features.

5. The log anomaly detection method according to any one of claims 1 to 4, characterized in that: The step of performing feature weight adjustment processing on the fused feature information, and performing log anomaly detection based on the fused feature information after the feature weight adjustment processing, includes: Perform linear transformation on the fused feature information to generate a query vector, a key vector, and a value vector: in, represents the query vector, represents the key vector, represents a value vector, represents the fusion feature information, represents the weight matrix for generating query vectors, represents the weight matrix for generating the key vector, The weight matrix representing the generated value vector; The query vector, the key vector and the value vector are weighted averaged, and the fused feature information is subjected to feature weight adjustment processing based on the weighted average result to obtain the fused feature information after the feature weight adjustment processing: in, represents the dot product of vectors, represents the index of the feature dimension, It is dimensional attention vector, Indicates position offset, Represents a value vector In Location The row vector of It represents the fusion feature information after feature weight adjustment. It is used to express conditions or constraints. Represents the query vector In Location No. The elements of the feature dimension, Represents the key vector No. feature dimensions, Represents the key vector In Location No. The elements of the feature dimension, Represents the position deviation matrix Middle position With location Position deviation between Log anomaly detection is performed based on the fused feature information after the feature weight adjustment process.

6. A log anomaly detection device, characterized in that: The log anomaly detection device comprises: The data processing module is used to pre-process the collected raw log data to obtain structured log data; A data parsing module, used to extract log semantic information, timestamp information and log key information from the structured log data, and determine the log time interval based on the timestamp information; A time feature processing module, used to generate time coding data according to the log semantic information and the log time interval, and perform feature enhancement processing on the time coding data to obtain time feature information; A log key feature processing module, used to generate log key encoding data based on the log key information, and perform feature enhancement processing on the log key encoding data to obtain log key feature information; A feature fusion module, used to fuse the time feature information with the log key feature information to obtain fused feature information; A log anomaly detection module, used to perform feature weight adjustment processing on the fused feature information, and perform log anomaly detection based on the fused feature information after the feature weight adjustment processing; The time feature processing module is further used to encode the log time interval through a time encoder to obtain initial time encoding data: in, It's time No. The initial time encoding data of the moment vector of dimensions, is a periodic activation function, wherein the periodic activation function includes a sine function, and represents the learnable encoding parameters; Encoding the log semantic information through a text encoder to obtain text encoding data; The initial time coding data and the text coding data are coded and merged to obtain time coding data; Comparing the performance of a teacher model and a student model in a pre-built Transformer encoder, and optimizing the Transformer encoder based on the performance comparison result, the Transformer encoder comprising a plurality of Transformer layers, the Transformer layers comprising a multi-head self-attention relationship distillation module, the multi-head self-attention relationship distillation module comprising the teacher model and the student model; The time-coded data is input into the optimized Transformer encoder for feature enhancement processing to obtain time feature information.

7. A log anomaly detection device, characterized in that: The log anomaly detection device comprises: a memory, a processor, and a log anomaly detection program stored in the memory and executable on the processor, wherein the log anomaly detection program is configured to implement the log anomaly detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a log anomaly detection program, and when the log anomaly detection program is executed by a processor, the log anomaly detection method according to any one of claims 1 to 5 is implemented.

9. A computer program product, characterized in that The computer program product comprises a log anomaly detection program, and when the log anomaly detection program is executed by a processor, the steps of the log anomaly detection method according to any one of claims 1 to 5 are implemented.

Images