A log anomaly detection method based on structure-aware heterogeneous graph transformer

Abstract

The present application relates to the technical field of log anomaly detection, in particular to a log anomaly detection method based on structure perception heterogeneous graph Transformer. The present application adopts Drain algorithm to parse original logs, extracts log events and component information, and constructs a directed weighted log heterogeneous graph containing node types, edge types and edge weights; uses a pre-trained language model BERT to perform context perception semantic coding on each node; constructs a structure perception heterogeneous graph Transformer network, dynamically modulates the heterogeneous mutual attention calculation and message passing process through a structure anomaly perception factor; adopts a joint loss function composed of hyperspherical minimum volume constraint and orthogonal regularization for model training, and uses the Euclidean distance between the graph-level deep representation and the feature space center as the anomaly score to complete detection. The present application can effectively improve the accuracy, robustness and generalization performance of log anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of log anomaly detection technology, specifically to a log anomaly detection method based on structure-aware heterogeneous graph Transformer. Background Technology

[0002] System logs are core data that records the operational status of complex distributed systems. Quickly and accurately detecting abnormal behavior from massive logs is an important means of ensuring system reliability and security.

[0003] Existing deep learning-based log anomaly detection methods suffer from three main shortcomings: First, most existing graph methods construct logs as isomorphic graphs, failing to distinguish between log event nodes and system component nodes. This makes it difficult to differentiate the evolutionary relationships within components and the interactions between components, hindering the accurate characterization of the intrinsic connections between multi-source heterogeneous data in complex systems. Second, existing graph attention networks rely solely on the semantic similarity of node features when calculating attention weights, neglecting statistical patterns in graph structures such as edge co-occurrence frequency and node degree distribution. This makes it difficult for the model to proactively identify low-frequency but destructive abnormal structural patterns, resulting in insufficient sensitivity to structural anomalies. Third, many methods still use static word vectors for node encoding, making it difficult to handle polysemous words and contextual dependencies in log text. Furthermore, traditional loss functions lack constraints on the compactness and orthogonality of the feature space, leading to severe redundancy between feature dimensions and limiting the model's discriminative ability and generalization performance. Summary of the Invention

[0004] This invention provides a log anomaly detection method based on structure-aware heterogeneous graph Transformer, which improves the accuracy and generalization ability of log anomaly detection in complex systems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a log anomaly detection method based on structure-aware heterogeneous graph Transformer, comprising: S100: Parse the raw logs and extract log events and log component information; S200: Construct a log heterogeneous graph based on the temporal relationship of log events, treat log events as nodes and assign them to component source types, treat the temporal transfer relationship between adjacent log events as directed weighted edges, and distinguish the edges into internal component interaction edges and cross-component interaction edges according to whether the source node and the target node belong to the same log component. S300: A pre-trained language model is used to perform context-aware semantic encoding on the log text corresponding to each node in the log heterogeneous graph to obtain the initial feature representation of the node; S400: Construct a structure-aware heterogeneous graph Transformer network, calculate a structure anomaly perception factor for each edge in the log heterogeneous graph; introduce the structure anomaly perception factor into the heterogeneous mutual attention calculation and heterogeneous message passing process respectively, dynamically modulate the information aggregation, and obtain the graph-level depth representation of the log heterogeneous graph through multi-layer stacking propagation; S500: The mean of the graph-level depth representation of the heterogeneous log graph in the training set is used as the center of the feature space, and the model is trained using a joint loss function consisting of hyperspherical minimum volume constraint and orthogonal regularization. S600: Input the heterogeneous log graph to be detected into the trained model, and use the distance between the graph-level depth representation of the heterogeneous log graph to be detected and the center of the feature space as the anomaly score. When the anomaly score exceeds the preset threshold, it is judged as an anomaly.

[0006] As a preferred embodiment of the present invention, in S200, the edge weight of the directed weighted edge is the number of times the log event pair appears in the log group.

[0007] As a preferred embodiment of the present invention, in S300, the context-aware semantic encoding step includes: Using the log text corresponding to the node as input, the word embedding, fragment embedding and position embedding are added element by element to construct the initial input encoding of the node; The initial input is encoded into a pre-trained language model with a bidirectional Transformer structure. The contextual dependencies between words in the log text are modeled through a self-attention mechanism, and the initial feature representation of the nodes is output.

[0008] As a preferred embodiment of the present invention, in S300, the pre-trained language model is a BERT model.

[0009] As a preferred embodiment of the present invention, in step S400, the calculation step of the structural anomaly perception factor includes: Obtain the co-occurrence count of log event pairs and the degree between the source node and the target node in the log heterogeneity graph; The co-occurrence count and the degrees of the source and target nodes are input into a learnable nonlinear mapping function, and after numerical range constraints are imposed by an activation function, a structural anomaly perception factor is output.

[0010] As a preferred embodiment of the present invention, in step S400, the step of introducing the structural anomaly perception factor into the heterogeneous mutual attention calculation and heterogeneous message passing process respectively includes: The target node is mapped to a query vector, and the source node connected to the target node is mapped to a key vector. Based on the multi-head attention mechanism, the structural anomaly perception factor and the transformation matrix related to the edge type are jointly introduced into the attention calculation. After normalization, the heterogeneous mutual attention weight of each edge is obtained. The source node features are mapped to the message vector based on the edge type-related linear projection matrix. The structural anomaly perception factor is fused with the message vector to obtain heterogeneous messages.

[0011] As a preferred embodiment of the present invention, in S400, the step of obtaining the graph-level depth representation of the log heterogeneous graph through multi-layer stacking propagation includes: When each layer of nodes is updated, a linear transformation corresponding to the node type is applied to the aggregated representation of the target node, and a residual connection is introduced to preserve the node representation of the previous layer. After completing multi-layer propagation, the final representations of all nodes in the log heterogeneous graph are aggregated to obtain a graph-level depth representation.

[0012] As a preferred embodiment of the present invention, in S500, the step of setting the minimum volume constraint of the hyperspherical surface includes: Calculate the mean of the graph-level depth representation of the log heterogeneous graph in the training set, and use it as the center of the feature space; A hypersphere minimum volume constraint term is constructed using the distance between the graph-level depth representation of the log heterogeneous graph in the training set and the center of the feature space. Minimizing the value of the constraint term is used as the optimization objective for model training.

[0013] As a preferred embodiment of the present invention, in S500, the orthogonal regularization step includes: Constraints are imposed on the product of the trainable parameter matrix of each layer in the model and its transpose, and an orthogonal regularization constraint term is constructed based on the difference between the product and the identity matrix. The orthogonal regularization constraint term is weighted by the regularization weight coefficient and added to the hypersphere minimum volume constraint term to form a joint loss function.

[0014] In a preferred embodiment of the present invention, in S600, the distance is a Euclidean distance.

[0015] The beneficial effects of this invention are: 1. This invention constructs logs as directed weighted heterogeneous graphs containing node types and edge types, distinguishing between intra-component interaction edges and cross-component interaction edges. It breaks through the limitations of existing methods in isomorphic graph modeling, can accurately depict the intrinsic relationships of multi-source heterogeneous log data in complex systems, and significantly improves the expressive power of log structures.

[0016] 2. This invention proposes a structural anomaly perception factor, which integrates the co-occurrence frequency and node degree deviation information of log event pairs, and simultaneously introduces them into the two stages of heterogeneous mutual attention calculation and message passing for multiplicative modulation. This enables the model to actively identify and strengthen low-frequency but destructive abnormal structural patterns, thus making up for the shortcomings of existing graph methods in perceiving structural anomalies.

[0017] 3. This invention designs a joint loss function that combines the minimum volume constraint of the hypersphere with orthogonal regularization. While promoting the compact aggregation of normal log graphs, it constrains the orthogonality of the parameter space, effectively reducing feature redundancy and enhancing the independence and discriminability of representations, thereby improving the model's generalization ability and detection accuracy on highly redundant log data. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a log anomaly detection method based on a structure-aware heterogeneous graph Transformer according to the present invention. Detailed Implementation

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] Example 1: As Figure 1 As shown, the present invention provides a log anomaly detection method based on structure-aware heterogeneous graph Transformer, comprising: S100: Parse the raw logs and extract log events and log component information; Specifically, the raw logs are unstructured text, and each log message typically consists of fields such as timestamp, log level, component to which it belongs, and message body. Due to the large number of raw logs and their inconsistent formats, directly processing the raw log text makes it difficult to effectively extract structured information; therefore, it is necessary to parse them first.

[0021] This step uses the Drain algorithm to parse the raw logs online. The Drain algorithm maintains a fixed-depth prefix tree, grouping log messages with similar structures into the same log event template, and replacing changing dynamic parameters with wildcards "<*>", thus mapping each raw log message to a corresponding log event. After parsing, a structured log event set is obtained. ,in Indicates the first A log event template This represents the total number of log events.

[0022] At the same time, the log component fields to which each original log message belongs are extracted to obtain the log component set. ,in Indicates the first A log component. This represents the total number of log components. A log component identifies the system module that generated the log message, such as dfs.DataNodeDataXceiver or dfs.FSNamesystem in the HDFS system. This identifier is used in subsequent step S200 to identify the component source type of each log event node and to distinguish between intra-component interaction edges and cross-component interaction edges.

[0023] S200: Construct a log heterogeneous graph based on the temporal relationship of log events, treat log events as nodes and assign them to component source types, treat the temporal transfer relationship between adjacent log events as directed weighted edges, and distinguish the edges into internal component interaction edges and cross-component interaction edges according to whether the source node and the target node belong to the same log component. Furthermore, in S200, the edge weight of the directed weighted edge is the number of times the log event pair appears in the log group.

[0024] Specifically, firstly, log identifiers such as BlockID and SessionID are used to divide the parsed log messages into several log groups, and log messages in the same log group correspond to the same system execution process.

[0025] Extract all unique log events from each log group, and define each log event as a node in the graph, forming a node set. Based on the log component information extracted in step S100, determine the log component category to which each log event node belongs, and construct a node type set. And through the mapping function Assign a corresponding node type to each node to identify the component source of the log event.

[0026] After the nodes are built, a set of directed edges is constructed according to the order in which the log events occur in the log group. Each directed edge is represented as ,in, For directed edges, As the source node, For the target node, when the log event template Closely follows the log event template in time. When it occurs, establish a [system / mechanism] point to The directed edge, This refers to the edge weight. For log events The number of co-occurrences throughout the entire log group is used to characterize the frequency characteristics of transitions between log events, reflecting the statistical regularities in the evolution of log events. For the source node The corresponding log event template, For the target node The corresponding log event template. Taking HDFS logs as an example, if log event LsE is generated by dfs.DataNode and LtE is generated by dfs.FSNamesystem, and this event pair appears 3 times in the log group, then a cross-component interaction edge with a weight of 3 is established. .

[0027] Then, based on the log component categories of the source and target nodes connected by the edge, the type attribute of the edge is determined, and an edge type set is constructed. When the source node and target node originate from the same logging component, the edge is marked as an intra-component interaction edge; when the source node and target node originate from different logging components, the edge is marked as a cross-component interaction edge. This is achieved through a mapping function. Assign a corresponding edge type to each edge. This design enables the model to differentiate between the evolutionary relationships within components and the interaction relationships between components, thereby more accurately characterizing the intrinsic relationships of multi-source heterogeneous data in complex systems.

[0028] Combining the above set of nodes, edges, node types, and edge types, each log group is modeled as a directed weighted heterogeneous graph. For any edge Its relational characteristics are determined by triples. Characterization is performed to describe the source components of log events and their interaction types. This process is repeated for each log group, ultimately resulting in a set of log heterogeneous graphs. This provides structured input for subsequent steps S300 and S400, where For the first A directed weighted heterogeneous graph corresponding to each log group. This represents the total number of log heterogeneous graphs.

[0029] S300: A pre-trained language model is used to perform context-aware semantic encoding on the log text corresponding to each node in the log heterogeneous graph to obtain the initial feature representation of the node; Further, in S300, the context-aware semantic encoding step includes: Using the log text corresponding to the node as input, the word embedding, fragment embedding and position embedding are added element by element to construct the initial input encoding of the node; The initial input is encoded into a pre-trained language model with a bidirectional Transformer structure. The contextual dependencies between words in the log text are modeled through a self-attention mechanism, and the initial feature representation of the nodes is output.

[0030] Furthermore, in S300, the pre-trained language model is a BERT model.

[0031] Specifically, in step S300, the pre-trained language model BERT is used to perform context-aware semantic encoding on the log text corresponding to each node in the log heterogeneous graph to obtain the initial feature representation of the node.

[0032] For log heterogeneous graphs For each log event node, its corresponding original log text is used as input. After the log text is lexicalized, three types of embedding vectors are constructed: word embedding, fragment embedding, and position embedding. The initial input code of the node is obtained by adding the three vectors element by element. ,in Indicates the first Vector representation of each word element This represents the total number of terms in the log text. Word embeddings are used to represent the semantic information of terms, fragment embeddings are used to distinguish the categories of text fragments, and positional embeddings are used to identify the positional information of terms in the sequence. The three types of embeddings characterize terms from the three dimensions of semantics, fragments, and positions, respectively. After adding them element by element, an initial input code that integrates multidimensional information is formed, which serves as the input for the subsequent pre-trained language model encoding.

[0033] The BERT model receives the above initial input encoding sequence. After being processed by stacked Transformer modules, the feature sequence is updated by simultaneously capturing the contextual dependencies between each word and all other words in the sequence through a bidirectional self-attention mechanism, making full use of the bidirectional contextual information of the log text. The feature sequence is used as the initial feature representation of the corresponding node in the log heterogeneous graph to achieve context-aware semantic encoding of the node.

[0034] Compared to static word vectors, the BERT model can effectively handle polysemous words and contextual dependencies in log text, providing higher semantic quality node feature inputs for graph structure learning in the subsequent step S400.

[0035] S400: Construct a structure-aware heterogeneous graph Transformer network, calculate a structure anomaly perception factor for each edge in the log heterogeneous graph; introduce the structure anomaly perception factor into the heterogeneous mutual attention calculation and heterogeneous message passing process respectively, dynamically modulate the information aggregation, and obtain the graph-level depth representation of the log heterogeneous graph through multi-layer stacking propagation; Furthermore, in S400, the calculation steps of the structural anomaly sensing factor include: Obtain the co-occurrence count of log event pairs and the degree between the source node and the target node in the log heterogeneity graph; The co-occurrence count and the degrees of the source and target nodes are input into a learnable nonlinear mapping function, and after numerical range constraints are imposed by an activation function, a structural anomaly perception factor is output.

[0036] Specifically, in step S400, a Structure-Aware Heterogeneous Graph Transformer (SA-HGT) network is constructed to perform deep representation learning on the log heterogeneous graph. SA-HGT adopts a layered stacked architecture, progressively learning the high-order structure and semantic features of the log heterogeneous graph through multi-layer propagation. The node update process at each layer includes three stages: First, based on node features and structural anomaly awareness factors, structurally aware heterogeneous mutual attention calculation is performed to determine the importance weights of neighboring source nodes to the target node; second, the structural anomaly awareness factors are introduced into the message passing process, and the source node features are multiplicatively modulated to generate a structurally modulated heterogeneous message; finally, a linear transformation corresponding to the node type is applied to the weighted aggregated intermediate representation, and residual connections are introduced to complete the target node-specific aggregation and feature update. The overall update mechanism is as follows: ; in, For the target node In the Layer feature representation; For the target node The set of neighboring source nodes; To connect the source node With the target node The edge weight is The set of directed edges; For the normalized source node Jingbian For the target node Heterogeneous mutual attention weights; For the source node Jingbian To the target node The final heterogeneous message transmitted; This indicates that a weighted message from all neighboring source nodes is aggregated.

[0037] For each edge in the log heterogeneous graph Calculate the structural anomaly sensing factor This measures the degree of atypicality of the edge relative to the overall log structure. Specifically, it retrieves log event pairs. The number of times an edge co-occurs is equal to its weight. and source node With the target node Degree in log heterogeneity graph and The above three quantities are input into a learnable nonlinear mapping function. By learning the deviation patterns between co-occurrence frequency and node connection scale, low-frequency edge features that are structurally destructive or deviate from conventional connection patterns are identified, and then activated by a function. Output structural anomaly perception factor after applying numerical range constraints: ; in, For the edge The structural anomaly perception factor is used to measure the degree of structural atypicality of the edge. The edge weight is represented as the log event pair. The number of co-occurrences in the log group is derived from the statistical results when constructing the directed weighted edge in step S200; and The source nodes in the log heterogeneous graph constructed in step S200 are respectively With the target node The degree; It is a learnable nonlinear mapping function; This is the activation function used to constrain the numerical range of the output. The larger the value, the more the structural characteristics of the edge deviate from the statistical regularity of normal logs, and the more likely it is to carry abnormal information.

[0038] Furthermore, in S400, the steps of introducing the structural anomaly perception factor into the heterogeneous mutual attention calculation and heterogeneous message passing processes respectively include: The target node is mapped to a query vector, and the source node connected to the target node is mapped to a key vector. Based on the multi-head attention mechanism, the structural anomaly perception factor and the transformation matrix related to the edge type are jointly introduced into the attention calculation. After normalization, the heterogeneous mutual attention weight of each edge is obtained. The source node features are mapped to the message vector based on the edge type-related linear projection matrix. The structural anomaly perception factor is fused with the message vector to obtain heterogeneous messages.

[0039] After determining the structural anomaly sensing factors for each side, perform heterogeneous mutual attention calculation for structural sensing to identify different source nodes. For the target node Importance weights for target nodes. Through the linear transformation matrix corresponding to the node type Mapped to query vector , source node Through linear transformation matrix Mapped to key vector : ; ; in, For the source node In the Key vectors under each attention head; For the target node In the The query vector under each attention head; For the source node type The corresponding number A linear transformation matrix of ___ key vectors, where It is an abbreviation for Key. Projecting the source node features of type to the first A space of key vectors; To match the target node type The corresponding number A linear transformation matrix of query vectors, where For the abbreviation of Query, Projecting the target node features of type to the first A query vector space; and Source nodes With the target node In the Layer feature representation.

[0040] Based on the multi-head attention mechanism, structural anomaly perception factors The transformation matrix related to the edge type is jointly introduced into the attention calculation. The original attention score for each attention head is calculated as follows: ; in, For the first The original attention scores of each attention head, where It is an abbreviation for Attention. edge type The corresponding learnable transformation matrix is ​​used to capture the heterogeneous semantic relationships between pairs of different node types. For node feature dimensions, The number of attention heads in multi-head attention. Feature dimensions for each attention head; This is the normalized result for the edge weights. For the source node With the target node Edge weights between them This represents the maximum value of all edge weights in the current graph. The modulation term for introducing structural anomaly sensing information is defined as follows: ; in, For the meta-relation triplet The corresponding learnable scalar tensor is used to distinguish attention scales under different meta-relations; This is a multiplicative modulation term for the structural anomaly perception factor, which gives greater weight to edges with higher degrees of structural anomaly during the attention allocation stage. For node feature dimensions. For the target node. splicing all neighboring source nodes Each attention head outputs an element and is then normalized using Softmax to obtain the heterogeneous mutual attention weights for each edge: ; in, For the normalized source node Jingbian For the target node Heterogeneous mutual attention weights; Indicates the first To the The output vectors of each attention head are concatenated to obtain the node feature dimension. Attention score vector; Indicates at the target node Normalize across all neighboring source nodes so that the sum of all neighboring attention weights is 1. For attention The abbreviation for .

[0041] After completing the attention weight calculation, heterogeneous message passing with structure modulation is performed to achieve the source node. To the target node Information transmission strength control. The output vector of each message header is calculated as follows: ; in, For the first The output vector of each message header; The multiplicative modulation term of the structural anomaly perception factor amplifies the message to enhance the information transmission strength of the abnormal structural pattern. For the source node type The corresponding number The nth linear projection matrix maps the source node features to the nth linear projection matrix. A message vector space, where MSG is an abbreviation for Message; For the source node In the Layer feature representation; edge type The corresponding learnable matrix is ​​used to incorporate edge dependency characteristics. (Concatenation) From the message headers, we obtain the final heterogeneous message: ; in, For the source node Jingbian To the target node The final heterogeneous message transmitted; Indicates to The outputs of each message header are concatenated to obtain a dimension of [dimensional value]. The message vector.

[0042] After completing attention calculation and message passing, the target node... All source nodes in the neighborhood The final heterogeneous messages are weighted and aggregated based on their corresponding attention weights: ; in, For the target node In the The intermediate representation of the layer after weighted aggregation has not yet undergone linear transformation of node types and residual connections; For the normalized source node Jingbian For the target node Heterogeneous mutual attention weights; For the source node Jingbian To the target node The final heterogeneous message transmitted; Indicates to the source The weighted messages from all neighboring source nodes are aggregated.

[0043] Furthermore, in S400, the step of obtaining a graph-level depth representation of the log heterogeneous graph through multi-layer stacking propagation includes: When each layer of nodes is updated, a linear transformation corresponding to the node type is applied to the aggregated representation of the target node, and a residual connection is introduced to preserve the node representation of the previous layer. After completing multi-layer propagation, the final representations of all nodes in the log heterogeneous graph are aggregated to obtain a graph-level depth representation.

[0044] Specifically, the intermediate representation obtained from the aggregation Apply to node type Corresponding linear transformation And through a nonlinear activation function Feature transformation is performed, and residual connections are introduced to preserve the representation of nodes in the previous layer, preventing information attenuation when multiple layers are stacked. ; in, For the target node In the The final feature representation of the layer after residual update; This is the intermediate representation of the weighted aggregation obtained from the above formula; For node type The corresponding linear transformation matrix is ​​used to apply type-specific feature transformations to the aggregated representations of different types of nodes, where A is an abbreviation for Aggregation; It is a non-linear activation function; For the target node In the The feature representation of the layer is preserved through residual connections to prevent information decay when multiple layers are stacked.

[0045] By stacking The final representation of the nodes is obtained from the layered SA-HGT network. For log heterogeneous graphs The final representations of all nodes are aggregated to construct a graph-level depth representation corresponding to the original log sequence. This is used for loss function optimization in subsequent step S500 and anomaly detection in step S600.

[0046] S500: The mean of the graph-level depth representation of the heterogeneous log graph in the training set is used as the center of the feature space, and the model is trained using a joint loss function consisting of hyperspherical minimum volume constraint and orthogonal regularization. Furthermore, in S500, the step of the minimum volume constraint of the hyperspherical surface includes: Calculate the mean of the graph-level depth representation of the log heterogeneous graph in the training set, and use it as the center of the feature space; A hypersphere minimum volume constraint term is constructed using the distance between the graph-level depth representation of the log heterogeneous graph in the training set and the center of the feature space. Minimizing the value of the constraint term is used as the optimization objective for model training.

[0047] Furthermore, in S500, the orthogonal regularization step includes: Constraints are imposed on the product of the trainable parameter matrix of each layer in the model and its transpose, and an orthogonal regularization constraint term is constructed based on the difference between the product and the identity matrix. The orthogonal regularization constraint term is weighted by the regularization weight coefficient and added to the hypersphere minimum volume constraint term to form a joint loss function.

[0048] Specifically, first, obtain the graph-level depth representation of all normal log heterogeneous graphs in the training set obtained through step S400. Calculate their mean as the center of the feature space. : ; in, Center of the feature space; The total number of normal log heterogeneous graphs in the training set; For the first The graph-level depth representation of a normal log heterogeneous graph obtained through step S400; the summation range covers all data in the training set. A normal log heterogeneous graph. Based on this, a hyperspherical minimum volume constraint term is constructed. By minimizing the graph-level depth representation of all normal log graphs in the training set and the feature space center The distance between them causes the embeddings of normal log graphs to cluster compactly towards the center within the feature space: ; in, This is the minimum volume constraint term for the hypersphere; For the first A graph-level depth representation of a normal log heterogeneous graph; Center of the feature space; The square of the Euclidean norm of the vector; This represents the total number of normal log heterogeneous graphs in the training set. This is achieved by minimizing... This makes the embedding distribution of normal log graphs cluster as compactly as possible around the center of the feature space, so that abnormal log graphs can obtain higher anomaly scores during the detection phase due to their deviation from the center.

[0049] Meanwhile, to reduce feature redundancy generated during model training and enhance the independence of feature representations, an orthogonal regularization constraint term is introduced. Apply orthogonal constraints to the trainable parameter matrices of each layer in the model: ; in, These are orthogonal regularization constraints; These are regularization weight coefficients used to control the strength of orthogonal constraints; This represents the total number of layers in the SA-HGT network. For the first The trainable parameter matrix of a layer network; for The transpose of the matrix; for and Identity matrices of the same dimension; This is the Frobenius norm, which is the square root of the sum of the squares of all elements of the matrix. (By constraining...) This reduces redundancy between different feature dimensions, enhances the independence and discriminativeness of feature representations, thereby improving the model's generalization ability and preventing overfitting.

[0050] Finally, the minimum volume constraint term of the hypersphere is added to the orthogonal regularization constraint term to form the joint loss function: ; in, For the joint loss function; The minimum volume constraint term for the hypersphere constrains the compact aggregation of normal log graph embeddings; The orthogonality regularization constraint term constrains the orthogonality of the model parameters. By minimizing the above joint loss function, the orthogonality and feature diversity of the model parameters are enhanced while promoting a compact distribution of normal log graph embeddings, thereby improving the model's ability to detect abnormal log behavior and its generalization performance.

[0051] S600: Input the heterogeneous log graph to be detected into the trained model, and use the distance between the graph-level depth representation of the heterogeneous log graph to be detected and the center of the feature space as the anomaly score. When the anomaly score exceeds the preset threshold, it is judged as an anomaly.

[0052] Furthermore, in S600, the distance is a Euclidean distance.

[0053] Specifically, the log heterogeneity graph to be detected The graph-level depth representation is obtained by sequentially performing semantic encoding in step S300 and the SA-HGT network in step S400. The depth of this map level corresponds to the feature space center determined in step S500. The Euclidean distance between them is used as an anomaly score: ; in, Heterogeneous log graph to be detected Abnormal scores; for The graph-level depth representation obtained after processing by the S400SA-HGT network in step S400 is compared with that obtained during the training phase. They exist in the same feature space; The feature space center is determined by the mean of the normal log graph embedding in the training set in step S500; Euclidean distance ( Norm). When If the threshold is exceeded, the log heterogeneous graph is determined to be abnormal.

[0054] In summary, this invention constructs a complete log anomaly detection framework through steps S100 to S600. It uses the Drain algorithm to parse the original logs and model them as a directed weighted heterogeneous graph, overcoming the limitations of existing methods that rely on isomorphic graph modeling. By using the pre-trained language model BERT to achieve context-aware semantic encoding of nodes, it solves the problem of static word vectors being unable to handle polysemous words and contextual dependencies. Through a structural anomaly perception factor, it dynamically modulates information aggregation in both the heterogeneous mutual attention calculation and message passing stages, enabling the model to proactively perceive low-frequency abnormal structural patterns. The joint loss function, composed of hyperspherical minimum volume constraints and orthogonal regularization, effectively reduces feature redundancy and enhances feature discriminability while promoting the compact embedding and aggregation of normal log graphs, thereby improving the accuracy and generalization performance of log anomaly detection.

[0055] Example 2: This example uses the actual running logs of HDFS, i.e., the Hadoop Distributed File System, to further illustrate the application effect of the present invention in real complex system scenarios.

[0056] During operation, the HDFS system generates a large number of log messages concurrently from multiple components such as dfs.DataNode, dfs.FSNamesystem, and dfs.DataNodeDataXceiver. Frequent cross-module interactions exist between these components, resulting in complex temporal relationships and significant heterogeneity in log events. Existing isomorphic graph-based detection methods treat all log event nodes as the same type, failing to distinguish semantic differences between log events generated by different components or to differentiate the internal evolution and cross-component interactions within components. This invention constructs a directed weighted log heterogeneous graph in step S200, assigning each node a component source type, such as dfs.DataNode, dfs.FSNamesystem, and dfs.DataNodeDataXceiver. Edges are categorized into internal component interaction edges and cross-component interaction edges, using the co-occurrence frequency of log event pairs as edge weights. This approach accurately characterizes the heterogeneous relationship structure between log events in concurrent HDFS multi-component scenarios.

[0057] This invention constructs a structure-aware heterogeneous graph Transformer network in step S400. Each layer of this network sequentially performs three stages: heterogeneous mutual attention calculation, heterogeneous message passing, and target node-specific aggregation. It calculates the structural anomaly perception factor for each edge, integrates log event pair co-occurrence frequency and node degree deviation information, and performs multiplicative modulation on it in the attention calculation and message passing stages. This enables the model to actively capture such low-frequency but destructive abnormal structural patterns, significantly improving the sensitivity to structural anomaly detection.

[0058] Furthermore, HDFS log data contains a large number of duplicate templates and highly similar event sequences, easily leading to severe redundancy across different dimensions in the feature space. This invention employs a joint loss function consisting of a hyperspherical minimum volume constraint and orthogonal regularization in step S500 for training. This not only encourages normal log graph embeddings to compactly cluster towards the center of the feature space but also constrains the orthogonality of the parameter matrices at each layer, effectively reducing feature redundancy and enhancing the separability of normal and abnormal log graphs in the feature space. Consequently, abnormal log graphs can be assigned higher anomaly scores during the detection phase, achieving accurate judgment.

[0059] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A log anomaly detection method based on structure-aware heterogeneous graph Transformer, characterized in that, include: S100: Parse the raw logs and extract log events and log component information; S200: Construct a log heterogeneous graph based on the temporal relationship of log events, treat log events as nodes and assign them to component source types, treat the temporal transfer relationship between adjacent log events as directed weighted edges, and distinguish the edges into internal component interaction edges and cross-component interaction edges according to whether the source node and the target node belong to the same log component. S300: A pre-trained language model is used to perform context-aware semantic encoding on the log text corresponding to each node in the log heterogeneous graph to obtain the initial feature representation of the node; S400: Construct a structure-aware heterogeneous graph Transformer network and calculate the structure anomaly perception factor for each edge in the log heterogeneous graph; The structural anomaly perception factor is introduced into the heterogeneous mutual attention calculation and heterogeneous message passing process respectively, and the information aggregation is dynamically modulated. The graph-level depth representation of the log heterogeneous graph is obtained through multi-layer stacking propagation. S500: The mean of the graph-level depth representation of the heterogeneous log graph in the training set is used as the center of the feature space, and the model is trained using a joint loss function consisting of hyperspherical minimum volume constraint and orthogonal regularization. S600: Input the heterogeneous log graph to be detected into the trained model, and use the distance between the graph-level depth representation of the heterogeneous log graph to be detected and the center of the feature space as the anomaly score. When the anomaly score exceeds the preset threshold, it is judged as an anomaly.

2. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S200, the edge weight of the directed weighted edge is the number of times the log event pair appears in the log group.

3. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S300, the context-aware semantic encoding step includes: Using the log text corresponding to the node as input, the word embedding, fragment embedding and position embedding are added element by element to construct the initial input encoding of the node; The initial input is encoded into a pre-trained language model with a bidirectional Transformer structure. The contextual dependencies between words in the log text are modeled through a self-attention mechanism, and the initial feature representation of the nodes is output.

4. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 3, characterized in that, In S300, the pre-trained language model is the BERT model.

5. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S400, the calculation steps for the structural anomaly sensing factor include: Obtain the co-occurrence count of log event pairs and the degree between the source node and the target node in the log heterogeneity graph; The co-occurrence count and the degrees of the source and target nodes are input into a learnable nonlinear mapping function, and after numerical range constraints are imposed by an activation function, a structural anomaly perception factor is output.

6. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S400, the steps of introducing the structural anomaly perception factor into the heterogeneous mutual attention calculation and heterogeneous message passing process respectively include: The target node is mapped to a query vector, and the source node connected to the target node is mapped to a key vector. Based on the multi-head attention mechanism, the structural anomaly perception factor and the transformation matrix related to the edge type are jointly introduced into the attention calculation. After normalization, the heterogeneous mutual attention weight of each edge is obtained. The source node features are mapped to the message vector based on the edge type-related linear projection matrix. The structural anomaly perception factor is fused with the message vector to obtain heterogeneous messages.

7. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S400, the step of obtaining a graph-level depth representation of the log heterogeneous graph through multi-layer stacking propagation includes: When each layer of nodes is updated, a linear transformation corresponding to the node type is applied to the aggregated representation of the target node, and a residual connection is introduced to preserve the node representation of the previous layer. After completing multi-layer propagation, the final representations of all nodes in the log heterogeneous graph are aggregated to obtain a graph-level depth representation.

8. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S500, the steps for the minimum volume constraint of the hypersphere include: Calculate the mean of the graph-level depth representation of the log heterogeneous graph in the training set, and use it as the center of the feature space; A hypersphere minimum volume constraint term is constructed using the distance between the graph-level depth representation of the log heterogeneous graph in the training set and the center of the feature space. Minimizing the value of the constraint term is used as the optimization objective for model training.

9. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 8, characterized in that, In S500, the orthogonal regularization step includes: Constraints are imposed on the product of the trainable parameter matrix of each layer in the model and its transpose, and an orthogonal regularization constraint term is constructed based on the difference between the product and the identity matrix. The orthogonal regularization constraint term is weighted by the regularization weight coefficient and added to the hypersphere minimum volume constraint term to form a joint loss function.

10. The log anomaly detection method based on structure-aware heterogeneous graph Transformer according to claim 1, characterized in that, In S600, the distance is the Euclidean distance.

Images