Alarm log anomaly detection method based on multi-feature fusion

Abstract

The application provides an alarm log anomaly detection method based on multi-feature fusion, which collects log data from a large amount of network alarm log data, performs log analysis on the alarm log data to extract a log template, obtains a log time vector, a log semantic vector and a log index vector, performs an adaptive weighted connection operation, and obtains a final fused log feature vector; the fused log feature vector is input into a BERT model to perform a hierarchical masking prediction task, a prediction result is obtained, and whether the alarm log is abnormal is judged according to an abnormal score threshold; the method provides a processing mode for multi-source heterogeneous log data, performs pattern recognition, data vector fusion and abnormal unified detection on the problem of complex structure in the massive alarm log data, and effectively improves the accuracy and efficiency of the alarm log anomaly detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network security log analysis, and particularly relates to an alarm log anomaly detection method based on multi-feature fusion. BACKGROUND

[0002] The scale and system of the industrial internet are expanding, and the intelligent and information levels are continuously improving. However, with the expansion of the protection boundary, new network attack methods and security risks are emerging. Although most industrial internet systems have deployed a large number of security monitoring and protection measures, the number of security alarms is still growing exponentially, and these alarms have the characteristics of various types and extensive sources, which brings unprecedented challenges to the safe and stable operation of the industrial internet.

[0003] The current parsing and processing of alarm logs in the Internet has the following problems: the alarm data is large, and manual analysis is time-consuming and laborious; most of the alarm data is false alarms or low-risk alarms, which greatly consumes the energy of security personnel; the alarm analysis and processing systems of various manufacturers are independent of each other, and security analysts need to process alarms generated by each manufacturer separately, which cannot be associated with analysis, and it is easy to miss high-risk events and difficult to implement traceability analysis. For multi-source heterogeneous logs, there are also the following problems: because the security manufacturers are independent of each other, the security logs generated by each alarm monitoring system have different structures and cannot be simply and uniformly processed.

[0004] The existing multi-source heterogeneous log processing technology mainly considers the template structure of log information, ignores the semantic information of log information, the time information between logs, and lacks the mining of deeper information of logs, thus lacking scalability for new log data. SUMMARY

[0005] The application provides an alarm log anomaly detection method based on multi-feature fusion, which uniformly represents by fusing the semantic, time, and index features of the log, to effectively identify anomalies in complex multi-source heterogeneous log data.

[0006] To achieve the above application purposes, the specific technical solutions are as follows:

[0007] The alarm log anomaly detection method based on multi-feature fusion comprises the following steps:

[0008] Step S1: Collecting alarm log data, performing log analysis on the alarm log data to extract a log template; according to the log time interval feature, the time interval between adjacent log messages is enhanced through linear transformation and attention mechanism to obtain a log time vector.

[0009] Step S2: extracting semantic information of the alarm log data based on the log template, obtaining a log semantic vector; using a hierarchical vocabulary table method to construct a dictionary tree, generating index information, and obtaining a log index vector.

[0010] Step S3: performing an adaptive weighted connection operation on the obtained log time vector, log semantic vector, and log index vector to obtain a final fused log feature vector.

[0011] Step S4: inputting the fused log feature vector into a BERT model to perform a hierarchical masking prediction task, obtaining a prediction result, and determining whether the alarm log is abnormal according to an abnormal score threshold.

[0012] Further, in step S1, the Drain method based on a fixed depth tree model is used to extract log templates from the alarm log data, and the specific process includes:

[0013] S101: preprocessing the obtained alarm log data, replacing substrings with regular expressions, replacing IP addresses, numbers, and email information variables with symbols "*";

[0014] S102: performing word segmentation on the preprocessed alarm log data to generate a word list ;

[0015] S103: initializing the root node of the parsing tree , log length , prefix word , and constructing a tree structure according to the word list ;

[0016] S104: checking whether the current log length exists in the tree, if it exists, updating the dictionary composed of the corresponding prefix word , if it does not exist, adding the current log length and the word list to the tree;

[0017] S105: calculating the similarity between the word segmentation sequence of each template in the log template library and each word segmentation sequence in the parsing tree, and selecting the template with the highest similarity as the log template .

[0018] Further, the method for obtaining the log time vector in step S1 includes:

[0019] extracting the timestamps and of each pair of adjacent log messages from the alarm log data, calculating the time interval of each pair of adjacent log messages, and obtaining log time interval features;

[0020] Converting the log time interval feature into an intermediate vector : ; wherein, represents a weight matrix, represents a bias vector, represents a dynamically weighted unit matrix;

[0021] Introducing an attention mechanism to obtain a log time vector : ;

[0022] ; wherein, , and are query, key and value vectors generated by the intermediate vector through different linear transformations; , and are linear transformation matrices, is the dimension of the key vector.

[0023] Further, the process of extracting the semantic information of the alarm log data to obtain the log semantic vector in step S2 using the Transformer model of the embedding contrast learning includes:

[0024] S201: Perform embedding representation on the word segmentation sequence in the log template T_m, and convert the word into a vector representation in a high-dimensional space, and the calculation formula is: ; wherein, is a word embedding function, is a position encoding function;

[0025] Select similar log message pairs as positive sample pairs, and select dissimilar log message pairs as negative sample pairs;

[0026] S202: Process the embedding representation of the log template through a multi-head self-attention layer , and the attention calculation is as follows:

[0027] ;

[0028] ;

[0029] ;

[0030] ; wherein, , , are respectively generated by The query, key, value matrix obtained by linear transformation, 、 、 is a linear transformation matrix, 、 、 and is a parameter matrix;

[0031] S203: further processing the output of self-attention by a feedforward network layer :

[0032] ; wherein, 、 is a weight matrix, 、 is a bias vector;

[0033] S204: applying a residual connection layer and a normalization representation to the final hidden state : ;

[0034] S205: optimizing the model using a contrastive learning loss function, the loss function is defined as:

[0035] ; wherein, represents the cosine similarity of two vectors, is a temperature parameter, and is a positive sample pair, and is a negative sample pair, is the number of negative samples;

[0036] S206: the calculation formula of the final log semantic vector is: ; wherein, is a function of average pooling operation on the hidden state, is a multilayer perceptron containing a hidden layer, is a projection matrix, is a bias vector.

[0037] Further, the log index vector extraction method includes:

[0038] S501: constructing a hierarchical vocabulary , grouping the words in the log template according to frequency and semantic correlation;

[0039] S502: establishing a dictionary tree structure each node represents a word or a group of words, and each branch from the root node to the leaf node represents a character sequence of a word;

[0040] S503: Assign a unique numerical identifier to each word in the dictionary tree and convert these identifiers into numerical vectors using word embedding techniques ;

[0041] S504: For each log entry, find the corresponding vectors in the dictionary tree through the words in its template , and generate the initial index vector for the log entry ; ;

[0042] S505: Use a hash function to enhance the uniqueness and discriminability of the index vector, and calculate the final log index vector :

[0043] ; where represents the element-level connection operation, T_m represents the log template, represents the hash function that maps the template string to a fixed-length numerical vector, represents the embedding function that converts the template into a semantic vector.

[0044] Further, the feature fusion method in step S3 includes:

[0045] Adaptive weighting and connection operation on three kinds of feature vectors: log semantic vector , log time vector and log index vector ;

[0046] Adaptive weight coefficient: where is the weight matrix, is the bias vector;

[0047] The final fused log feature vector is: .

[0048] Further, the hierarchical masking prediction task in step S4 includes:

[0049] Hierarchical masking on the final fused log feature vector: where , and are the parts of the fused vector corresponding to the log semantic feature, the log time feature, and the log index feature, respectively;​

[0050] Different masking ratios are applied to different parts, including the log time feature part. The masking ratio is 20%, and the log semantic feature part The masking ratio is 40%, log index feature part The shading rate is 30%;

[0051] The occlusion operation replaces the feature value of the selected location with This can be achieved using tags or random values.

[0052] Furthermore, in step S4, the BERT model is used to predict the occluded feature values, and the prediction probability is calculated:

[0053] The masked fusion vector Input the BERT model to obtain the feature representation of each location;

[0054] For the obscured location Calculate its prediction probability: ;in, Indicates position The original feature value at that location, Indicates that, apart from location All input feature values ​​other than those in the input, This is the weight matrix. It is the bias vector;

[0055] Calculate anomaly scores based on predicted probabilities. : ;in, It is the set of locations that are obscured. It represents the number of locations that are obscured.

[0056] When abnormal scores Exceeding the preset threshold When this happens, the corresponding log sequence will be marked as an anomaly.

[0057] Compared with the prior art, the beneficial effects of this invention are:

[0058] This invention solves the challenge of processing multi-source heterogeneous log data by using a multi-feature fusion approach. Based on a unified representation method of multi-feature fusion, it effectively processes multi-source heterogeneous log data, integrates semantic vectors, time vectors, and index vectors, and achieves comprehensive utilization of multi-dimensional features, thereby improving the comprehensiveness and accuracy of anomaly detection. Based on the BERT-based hierarchical masking prediction task, it adopts a differentiated masking ratio strategy, using different masking ratios for different feature parts, further enhancing the model's ability to perceive abnormal patterns. Attached Figure Description

[0059] Figure 1 is a multi-feature fusion-based alarm log anomaly detection method flowchart of the present application;

[0060] Figure 2 is an example of a parsing tree constructed when the Drain method is parsed; DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application are described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0062] It should be noted that the data sets used herein are derived from the log data sets collected in real scenarios existing in loghub, and three data sets are selected as experimental data sources, which are:

[0063] (1) HDFS: collected from more than 200 Amazon EC2 nodes running Hadoop-based jobs. It consists of 11,175,629 original log messages, 284,818 of which are labeled as abnormal.

[0064] (2) BGL: collected from the BlueGene / L supercomputer system, containing 4,747,963 original log messages, of which 348,469 are abnormal logs.

[0065] (3) Tbird: collected from the Thunderbird supercomputer system, and the first 20,000,000 log messages are selected to form the data set, of which 758,562 are abnormal.

[0066] The data set is large in size and diverse in abnormal types, which can well verify the ability of our method to process large-scale heterogeneous logs.

[0067] As shown in Figure 1 , it is a multi-feature fusion-based alarm log anomaly detection method flowchart of the present application, which clearly shows the complete process from log collection to anomaly detection. The method comprises the following steps:

[0068] Step S1: collect alarm log data, perform log parsing on the alarm log data to extract log templates; according to the log time interval feature, the time interval between adjacent log messages is enhanced through linear transformation and attention mechanism to obtain a log time vector.

[0069] The Drain method based on the fixed depth tree model is used for log analysis and extraction of log templates from the alarm log data, and the specific process includes:

[0070] S101: The acquired alarm log data is preprocessed, and regular expressions are used to replace substrings, and IP addresses, numbers, and email information variables are replaced with symbols "*".

[0071] We designed a set of regular expression patterns for common variable types: IP address pattern , timestamp pattern , MAC address pattern , UUID pattern and the like. For logs in specific fields, custom patterns such as system path / [\w / \.]+ and process ID:pid=\d+ are also added. The preprocessing accuracy of these patterns reached 96.7% in our tests, greatly improving the efficiency of subsequent template extraction. For example: r"(?<=blk_)[-\d]+": matches a number or negative number starting with blk_, r'\d+\.\d+\.\d+\.\d+': matches IP addresses and replaces them with wildcard *. The construction process of the parse tree is refined layer by layer, and through the tree structure, similar logs can be efficiently grouped and common templates can be extracted, greatly reducing the computational complexity.

[0072] As Figure 2 shown, is an example of a parse tree constructed by the Drain method of the present application during analysis, showing the tree structure in the log analysis process; the figure shows a three-layer depth parse tree, with the root node located at the top, the first layer classified according to log length L_t (the figure shows four branches with lengths of 16, 11, 10, and 9), the second layer subdivided according to prefix words (such as the keywords Receiving and Verification), and the third layer containing specific word lists and corresponding log templates . The right branch of the figure shows a complete path: starting from the root node, passing through the branch with length 11, to the node with prefix word Receiving, and finally to the leaf node containing the complete word list, and the corresponding log template Receiving block * of size * from is extracted; the entire tree structure is organized hierarchically, enabling efficient log template matching and extraction.

[0073] S102: Tokenize the preprocessed alarm log data to generate a word list The word segmentation process employs a Unicode-based text segmentation algorithm, supporting multilingual log processing. For English logs, spaces and punctuation are used as delimiters; for Chinese logs, the jieba word segmentation library is integrated; for logs with special formats such as JSON or XML structures, a dedicated parser is used to extract key-value pairs. The segmentation results are filtered for stop words, removing words like "the," "a," and "an" that have no substantial meaning for template extraction, while retaining keywords indicating status such as "error," "failed," and "warning," improving the template's discriminative ability.

[0074] S103: Initialize the root node of the parse tree Log length prefix words And based on the word list A tree structure is constructed. The maximum depth of the parse tree is set to 5 levels, which is the optimal parameter verified through extensive experiments, achieving a good balance between processing efficiency and template accuracy. The root node stores global statistics. The first level groups logs by length (short sentences <10 words, medium sentences 10-20 words, long sentences >20 words). The second level groups by prefix words (taking the first 1-2 non-variable words from the log). Subsequent levels are further subdivided based on keywords at specific positions.

[0075] S104: Check the current log length If a word exists in the tree, update the corresponding prefix word. If the dictionary does not exist, then the current log length will be used. and word list Add it to the tree.

[0076] S105: Calculate the similarity between the word segmentation sequence of each template in the log template library and the word segmentation sequence in the parse tree, and select the template with the highest similarity as the log template. .

[0077] Methods for obtaining log time vectors include:

[0078] Extract the timestamp of each pair of adjacent log messages from the alarm log data. and Calculate the time interval for each pair of adjacent log messages. This yields the log time interval characteristics. Time interval The distribution is usually uneven and exhibits a long-tail effect, therefore we use logarithmic transformation. Normalization is performed.

[0079] weight matrix Initialize as a normal distribution N(0, 0.01) with dimension 1. ×1, where is the dimension of time vector, which is set to 64 in our experiment. Bias vector is initialized to zero vector. Dynamic weighting matrix is a diagonal matrix, whose diagonal elements are calculated by exponential decay attention mechanism, so that the model can pay more attention to the recent time interval pattern, and the decay rate is set to 0.9.

[0080] Convert the log time interval feature into an intermediate vector : ; wherein, denotes the weight matrix, denotes the bias vector, denotes the dynamically weighted identity matrix.

[0081] Introduce attention mechanism to obtain log time vector : .

[0082] ; wherein, , and are query, key and value vectors generated by the intermediate vector through different linear transformations; , and are linear transformation matrices, is the dimension of the key vector.

[0083] The Transformer model using embedding contrast learning is used to extract the semantic information of the alarm log data, and obtain the log semantic vector ; the process includes:

[0084] S201: Perform embedding representation on the segmented sequence in the log template , convert the word into vector representation in high-dimensional space, and the calculation formula is: ; wherein, is a word embedding function, is a position encoding function;

[0085] Select similar log message pairs as positive sample pairs, and dissimilar log message pairs as negative sample pairs; the word embedding adopts a pre-trained Word2Vec model, and the vector dimension is 300, which is trained on a large-scale log corpus by using a sliding window size of 5. For domain-specific vocabulary, we use transfer learning to fine-tune the embedding vector, so that the embedding vector can better express the semantic relationship in the network security field.

[0086] The position encoding adopts the sine and cosine function and , wherein, is the position of the word in the sequence, is the dimension index. The selection of positive sample pairs is based on logs within the same cluster, and the selection of negative sample pairs is random from different clusters, keeping the positive-negative sample ratio at 1:5.

[0087] S202: Process the embedding representation of the log template through a multi-head self-attention layer The attention calculation is as follows:

[0088] ;

[0089] ;

[0090] ;

[0091] ; wherein, , , are the query, key, and value matrices obtained by linear transformation, , , , are linear transformation matrices, , , and are parameter matrices;

[0092] S203: Further process the output of self-attention through a feedforward network layer :

[0093] ; wherein, , are weight matrices, , are bias vectors;

[0094] S204: Apply a residual connection layer and normalization to represent the final hidden state : ;

[0095] S205: Optimize the model using a contrastive learning loss function, defined as:

[0096] ; wherein, represents the cosine similarity of two vectors, is a temperature parameter, and are positive sample pairs, and are negative sample pairs, is the number of negative samples.

[0097] S206: final log semantic vector The calculation formula is: ; wherein, is a function of average pooling operation on the hidden state, is a multilayer perceptron containing a hidden layer, is a projection matrix, is a bias vector.

[0098] Step S2: extracting the semantic information of the alarm log data based on the log template, obtaining a log semantic vector; using a hierarchical vocabulary table to construct a dictionary tree, generating index information, and obtaining a log index vector.

[0099] Log index vector The extraction method comprises:

[0100] S501: constructing a hierarchical vocabulary table , grouping the words in the log template according to frequency and semantic correlation.

[0101] S502: establishing a dictionary tree structure , each node representing a word or word group, and each branch from the root node to the leaf node representing a character sequence of a word.

[0102] S503: assigning a unique numerical identifier to each word in the dictionary tree , and converting these identifiers into numerical vectors using word embedding technology.

[0103] S504: for each log entry, find the corresponding vector in the dictionary tree through the words in its template , and generate the initial index vector of the log entry.

[0104] S505: using a hash function to enhance the uniqueness and discriminability of the index vector, calculating the final log index vector :

[0105] ; wherein, represents an element-level connection operation, T_m represents a log template, represents a hash function that maps a template string to a fixed-length numerical vector, represents an embedding function that converts a template into a semantic vector.

[0106] The hash function adopts a local sensitive hash (LSH) algorithm, specifically a SimHash, which can map similar templates to a similar hash space, maintaining the similarity relationship between templates. The hash vector dimension is set to 128, the embedding vector dimension is 256, and the final connected index vector dimension is 384.

[0107] Step S3: performing an adaptive weighted connection operation on the obtained log time vector, log semantic vector and log index vector to obtain a final fused log feature vector.

[0108] The feature fusion method includes:

[0109] The three feature vectors: log semantic vector , log time vector and log index vector are subjected to an adaptive weighted and connected operation.

[0110] The adaptive weight coefficient is: wherein, is a weight matrix, and is a bias vector.

[0111] The dimension of the weight matrix is wherein, are the dimensions of the semantic vector, time vector and index vector, respectively.

[0112] Initialization adopts the Xavier method to ensure that each feature obtains similar weight attention at the beginning of training. During the training process, the weight distribution is automatically adjusted through back propagation, so that the model can flexibly adjust the feature importance according to different types of abnormal scenes. The semantic vector obtains a higher weight (about 0.5-0.6) in content anomaly detection, the time vector has a significant weight (about 0.4-0.5) in time sequence anomaly detection, and the index vector performs outstandingly (about 0.4-0.6) in structural anomaly detection.

[0113] The final fused log feature vector is: .

[0114] Step S4: inputting the fused log feature vector into a BERT model to perform a hierarchical masking prediction task, obtaining a prediction result, and determining whether the alarm log exists according to an abnormal score threshold.

[0115] The hierarchical masking prediction task includes:

[0116] The hierarchical masking of the final fused log feature vector is: wherein , and These are the parts of the fusion vector corresponding to log semantic features, log time features, and log index features, respectively.

[0117] Different masking ratios are applied to different parts, including the log time feature part. The masking ratio is 20%, and the log semantic feature part The masking ratio is 40%, log index feature part The occlusion ratio is 30%.

[0118] The occlusion operation replaces the feature value of the selected location with Masking is achieved using either labels or random values. The masking operation employs the following strategy: 80% of the selected locations are replaced with a special [MASK] label, 10% are replaced with random values ​​(sampled from a similar feature distribution), and 10% remain unchanged. This hybrid strategy avoids feature distribution shift issues during the inference phase, improving the model's generalization ability. Hierarchical random sampling is used to select masking locations, ensuring a uniform distribution across all parts. Furthermore, the masking pattern is optimized using a Markov chain Monte Carlo method to prevent overly concentrated or dispersed masking, maximizing the model's learning efficiency.

[0119] In step S4, the BERT model is used to predict the occluded feature values, and the prediction probability is calculated:

[0120] The masked fusion vector Input the BERT model to obtain the feature representation of each location;

[0121] For the obscured location Calculate its prediction probability: ;in, Indicates position The original feature value at that location, Indicates that, apart from location All input feature values ​​other than those in the input, This is the weight matrix. This is the bias vector.

[0122] Calculate anomaly scores based on predicted probabilities. : ;in, It is the set of locations that are obscured. It represents the number of locations that are obscured.

[0123] When abnormal scores Exceeding the preset threshold When this happens, the corresponding log sequence will be marked as an anomaly.

[0124] threshold The determination of the threshold value adopts a dynamic threshold method, which is adaptively adjusted based on the statistical characteristics of the normal log distribution; in the specific implementation, we use the abnormal score distribution of the historical data to calculate the 95% quantile point as the basic threshold, and then combine the exponential weighted moving average (EWMA) method to track the trend of the score distribution, so as to realize the dynamic adjustment of the threshold value.

[0125] The system also supports time window-based threshold adjustment, and different threshold strategies are adopted for working days and non-working hours, which solves the problem of insufficient time sensitivity of the traditional fixed threshold. In the high security requirement scene, the threshold value can be set to the 90% quantile point to improve the recall rate; in the low false alarm requirement scene, it can be set to the 99% quantile point to improve the precision. Experiments show that the dynamic threshold value can improve the F1 score by an average of 8.3% compared with the fixed threshold value.

[0126] It should be noted that the log parsing method enhanced by semantic segmentation can also be further used in step S1:

[0127] S901: using a pre-trained language model to perform preliminary semantic understanding on the log message to obtain a semantic representation ;

[0128] S902: based on the semantic representation to identify the variable part and the template part in the log , and calculate the probability of each word belonging to the variable:

[0129] ; wherein, is a sigmoid function, is a weight matrix, is a bias vector, is the semantic representation of the i-th word ;

[0130] S903: generate a more accurate log template by combining the Drain method and the semantic segmentation result ;

[0131] S904: the parsing accuracy of the semantic segmentation enhancement is calculated by the following formula:

[0132] ; wherein, denotes the enhanced accuracy, denotes the correct template set, denotes the total template set, is the semantic enhancement weight coefficient, ​representing semantic confidence:

[0133] where, is the number of words in the log message.

[0134] The method also includes an incremental learning mechanism:

[0135] S1001: Periodically update the model using newly collected log data update the model parameters;

[0136] S1002: Keep the historical model core knowledge, while adapting to emerging log patterns, generate a new model ;

[0137] S1003: Use knowledge distillation techniques to preserve key information during model updates, distillation loss function is defined as:

[0138] where, represents the distillation loss, is the cross-entropy loss, is the KL divergence loss, is the balance coefficient, y is the true label, is the new model prediction value, is the old model prediction value, is the new model prediction value;

[0139] S1004: Determine whether the model needs to be updated by calculating the uncertainty of the new sample x:

[0140] where, represents the probability that the sample belongs to the class , represents the possible class set of the sample (normal or abnormal), when the uncertainty exceeds the threshold trigger model update.

[0141] ​In the production environment, the system also implements an incremental learning mechanism to update the model by continuously absorbing new log patterns and abnormal samples. The incremental update adopts a sliding window strategy, fine-tuning the latest data every 8 hours, while using the elastic weight consolidation (EWC) algorithm to prevent the catastrophic forgetting problem. The system is also equipped with an anomaly explanation module, which calculates the contribution of each feature to the abnormal judgment through SHAP (SHapley Additive exPlanations) values, generating human understandable explanations of the abnormal reasons, helping security analysts quickly locate the problem source. Experimental results show that this incremental learning mechanism improves the detection ability of the model by 23% when facing new attack patterns, significantly reducing the model maintenance cost and manual analysis time.

[0142] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting anomalies in alarm logs based on multi-feature fusion, characterized in that, The method includes the following steps: Step S1: Collect alarm log data, parse the alarm log data to extract log templates; based on the log time interval characteristics, enhance the time interval between adjacent log messages through linear transformation and attention mechanism to obtain the log time vector; Methods for obtaining log time vectors include: Extract the timestamp of each pair of adjacent log messages from the alarm log data. and Calculate the time interval for each pair of adjacent log messages. This yields the log time interval characteristics; Convert log time interval features into an intermediate vector : ;in, Represents the weight matrix. Represents the deviation vector. Represents the dynamically weighted identity matrix; Introducing an attention mechanism to obtain the log time vector : ; ;in, , and It is composed of intermediate vectors Query, key, and value vectors generated through different linear transformations; , and It is a linear transformation matrix. It is the dimension of the key vector; Step S2: Extract semantic information from alarm log data based on the log template to obtain log semantic vector; construct a dictionary structure tree using a hierarchical vocabulary approach to generate index information and obtain log index vector; Obtain log semantic vector The process includes: S201: Regarding log templates The word segmentation sequence is embedded to represent the words, converting them into vector representations in a high-dimensional space. The calculation formula is as follows: ;in, For word embedding functions, For position encoding functions; Similar log message pairs are selected as positive sample pairs, and dissimilar log message pairs are selected as negative sample pairs. S202: Embedded Representation of Log Templates Through Multi-Head Self-Attention Layer Attention is calculated as follows: ； ； ； ;in, , , They are respectively from The query, key, and value matrix obtained through linear transformation , , It is a linear transformation matrix. , , and For parameter matrices; S203: Further process the self-attention output through a feedforward network layer. : ;in, , This is the weight matrix. , It is the bias vector; S204: Applying residual connection layers and normalized representations to the final hidden state : ; S205: The model is optimized using a contrastive learning loss function, defined as follows: ;in, Cosine similarity represents the similarity between two vectors. It's a temperature parameter. and These are positive sample pairs. and These are negative sample pairs. The number of negative samples; S206: Final Log Semantic Vector The calculation formula is: ;in, It is a function that performs average pooling on the hidden state. It is a multilayer perceptron containing one hidden layer. It is a projection matrix. It is the bias vector; Log index vector Extraction methods include: S501: Constructing a hierarchical vocabulary , log template The words in the text are grouped according to frequency and semantic relevance; S502: Establish a dictionary structure tree Each node represents a word or group of words, and each branch from the root node to the leaf node represents a sequence of characters for a word. S503: For dictionary structure tree Each word in the list is assigned a unique numerical identifier. And use word embedding technology to convert these identifiers into numerical vectors. ; S504: For each log entry, via its template The words in the dictionary structure tree Find the corresponding numerical vector in Generate the initial index vector for this log entry. ; S505: Use a hash function to enhance the uniqueness and distinguishability of the index vector, and calculate the final log index vector. : ;in, This indicates an element-level join operation. This indicates a log template. This represents a hash function that maps a template string to a fixed-length numeric vector. This represents an embedding function that converts a template into a semantic vector; Step S3: Perform an adaptive weighted join operation on the obtained log time vector, log semantic vector, and log index vector to obtain the final fused log feature vector; Step S4: Input the fused log feature vector into the BERT model to perform a hierarchical masking prediction task, obtain the prediction result, and determine whether the alarm log is abnormal based on the anomaly score threshold; Use the BERT model to predict the occluded feature values ​​and calculate the prediction probability: The masked fusion vector Input the BERT model to obtain the feature representation of each location; For the obscured location Calculate its prediction probability: ;in, Indicates position The original feature value at that location, Indicates that, apart from location All input feature values ​​other than those in the input feature value, This is the weight matrix. It is the bias vector; Calculate anomaly scores based on predicted probabilities. : ;in, It is the set of locations that are obscured. It represents the number of locations that are obscured. When abnormal scores Exceeding the preset threshold When this happens, the corresponding log sequence will be marked as an anomaly.

2. The alarm log anomaly detection method based on multi-feature fusion according to claim 1, characterized in that, Step S1 uses the Drain method based on a fixed-depth tree model to parse and extract log templates from the alarm log data. The specific process includes: S101: Preprocess the acquired alarm log data, use regular expressions to replace substrings, and replace IP address, number and email information variables with the symbol "*". S102: Perform word segmentation on the preprocessed alarm log data to generate a word list. ; S103: Initialize the root node of the parse tree Log length prefix words And based on the word list Construct a tree structure; S104: Check the current log length If a word exists in the tree, update the corresponding prefix word. If the dictionary does not exist, then the current log length will be used. and word list Add to the tree; S105: Calculate the similarity between the word segmentation sequence of each template in the log template library and the word segmentation sequence in the parse tree, and select the template with the highest similarity as the log template. .

3. The alarm log anomaly detection method based on multi-feature fusion according to claim 1, characterized in that, The feature fusion method in step S3 includes: For the three feature vectors: log semantic vector Log time vector and log index vector Perform adaptive weighted summation and join operations; Adaptive weighting coefficients: ,in, This is the weight matrix. It is the bias vector; Final fused log feature vector for: .

4. The alarm log anomaly detection method based on multi-feature fusion according to claim 1, characterized in that, The layered occlusion prediction task in step S4 includes: Layered masking is applied to the final fused log feature vector: ,in , and These are the parts of the fusion vector corresponding to log semantic features, log time features, and log index features, respectively. Different masking ratios are applied to different parts, including the log time feature part. The masking ratio is 20%, and the log semantic feature part The masking ratio is 40%, log index feature part The shading rate is 30%; The occlusion operation replaces the feature value of the selected location with This can be achieved using tags or random values.

Images