Log detection method and device based on heterogeneous graph

Abstract

The invention provides a log detection method and device based on heterogeneous graphs, and the method comprises the steps: obtaining heterogeneous log data of a target service, and constructing at least two heterogeneous graphs containing multiple types of nodes based on the heterogeneous log data. And respectively inputting the at least two heterogeneous graphs into the heterogeneous graph model to obtain semantic alignment information respectively corresponding to the at least two heterogeneous graphs. And determining detection information of the heterogeneous log data based on the at least two pieces of semantic alignment information and the at least two heterogeneous graphs. On the basis of the characteristics of isomerization, unstructuring, high dimension, time sequence and the like of the log data, the heterogeneous graph is constructed on the basis of the heterogeneous log data, the heterogeneous graph can well reflect the isomerism and the time sequence of the log data, and the log data can be accurately described. Log detection is carried out based on the heterogeneous graph, and the accuracy of log detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a log detection method and device based on heterogeneous graphs. Background Art

[0002] With the development of high-performance architectures such as microservices, services in digital service networks can be accessed and removed quickly. This has led to a significant increase in the heterogeneity and complexity of digital service networks, making anomaly detection in these networks increasingly challenging. Logs, as system runtime information, are an important resource for understanding the status of various system infrastructures in complex digital service networks. By detecting logs, we can understand the status of various system infrastructures in complex digital service networks.

[0003] In the prior art, the abnormal log detection method based on graph neural network generally models the log as graph data, where the log is regarded as a node and the relationship between the nodes is regarded as an edge. The existing abnormal log detection method based on graph neural network is mainly performed in isomorphic graphs, and potential anomalies are identified by capturing the complex relationships and patterns between log nodes. However, due to the heterogeneity and complexity of log formats and semantics, this detection method ignores the inherent heterogeneity and temporal nature of log files, resulting in low accuracy of anomaly detection. Therefore, there is an urgent need for a more effective log detection method based on heterogeneous graphs to solve the above problems. Summary of the invention

[0004] In view of this, the embodiments of this specification provide a log detection method based on a heterogeneous graph. This specification also relates to a log detection device based on a heterogeneous graph, a computing device, a computer-readable storage medium and a computer program product to solve the above problems existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a log detection method based on a heterogeneous graph is provided, including: Obtaining heterogeneous log data of a target service, and constructing at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data; Inputting at least two heterogeneous graphs into the heterogeneous graph model respectively to obtain semantic alignment information corresponding to the at least two heterogeneous graphs respectively; Detection information of the heterogeneous log data is determined based on at least two pieces of semantic alignment information and the at least two heterogeneous graphs.

[0006] Optionally, the constructing at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data includes: Determine a log sequence corresponding to the heterogeneous log data, and extract at least two target logs from the log sequence; Based on the target node corresponding to each target log and the node type of the target node, heterogeneous graphs corresponding to the at least two target logs are constructed.

[0007] Optionally, the construction of a heterogeneous graph corresponding to any target log includes: Determine a target node corresponding to the target log and a node type of the target node; Determine a target node pair in the target node according to the node type, and determine a target edge of the target node pair; An initial heterogeneous graph is constructed based on the target node pair and the target edge, and time information is allocated to the nodes and edges in the initial heterogeneous graph to obtain a heterogeneous graph corresponding to the target log.

[0008] Optionally, determining the semantic alignment information of the target heterogeneous graph among the at least two heterogeneous graphs includes: Determine event nodes and non-event nodes included in the target heterogeneous graph; Processing the event node based on the left encoder in the heterogeneous graph model to obtain event information, and processing the non-event node based on the right encoder in the heterogeneous graph model to obtain non-event information; Semantically align the event information and the non-event information in an event representation space to obtain semantic alignment information.

[0009] Optionally, the determining the detection information of the heterogeneous log data based on the at least two pieces of semantic alignment information and the at least two heterogeneous graphs includes: Generate time dependency information corresponding to each heterogeneous graph based on at least two pieces of semantic alignment information, and determine an abnormal heterogeneous graph containing an abnormal node based on the at least two pieces of time dependency information and the at least two heterogeneous graphs; Detection information of the heterogeneous log data is determined based on the abnormal nodes included in the abnormal heterogeneous graph.

[0010] Optionally, the determining of the abnormal node includes: Determine a preset detection node, and respectively determine nodes to be detected in the at least two heterogeneous graphs; Based on at least two time-dependent information and the preset detection nodes, distance detection is performed on the nodes to be detected respectively corresponding to the at least two heterogeneous graphs, and the abnormal node is determined according to the detection result and the distance threshold.

[0011] Optionally, the generating the time dependency information corresponding to each heterogeneous graph based on the at least two pieces of semantic alignment information respectively includes: Performing intra-heterogeneous-graph encoding on the at least two heterogeneous graphs based on the at least two semantic alignment information to obtain heterogeneous information corresponding to each heterogeneous graph; The heterogeneous information corresponding to each heterogeneous graph is extracted to obtain the time-dependent information corresponding to each heterogeneous graph.

[0012] Optionally, the pre-training of the heterogeneous graph model includes: Determine a sample heterogeneous graph, input the sample heterogeneous graph into an initial heterogeneous graph model, and determine node alignment information of the sample heterogeneous graph through global alignment matching and local alignment matching; Determine a sample label of the sample heterogeneous graph, and adjust the parameters of the initial heterogeneous graph model based on the sample label, the node alignment information and the loss function until the heterogeneous graph model that meets the pre-training stop condition is obtained.

[0013] According to a second aspect of an embodiment of this specification, a log detection device based on a heterogeneous graph is provided, including: An acquisition module is configured to acquire heterogeneous log data of a target service and construct at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data; An input module, configured to input at least two heterogeneous graphs into the heterogeneous graph model respectively, and obtain semantic alignment information corresponding to the at least two heterogeneous graphs respectively; A determination module is configured to determine detection information of the heterogeneous log data based on at least two pieces of semantic alignment information and the at least two heterogeneous graphs.

[0014] According to a third aspect of an embodiment of this specification, a computing device is provided, including a memory, a processor, and a computer program or instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the heterogeneous graph-based log detection method when executing the computer program or instructions.

[0015] According to a fourth aspect of an embodiment of this specification, a computer-readable storage medium is provided, which stores a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the heterogeneous graph-based log detection method are implemented.

[0016] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-mentioned heterogeneous graph-based log detection method.

[0017] The log detection method based on heterogeneous graphs provided in this specification obtains heterogeneous log data of the target service, and constructs at least two heterogeneous graphs containing multi-type nodes based on the heterogeneous log data. The at least two heterogeneous graphs are respectively input into the heterogeneous graph model to obtain the semantic alignment information corresponding to the at least two heterogeneous graphs. The detection information of the heterogeneous log data is determined based on the at least two semantic alignment information and the at least two heterogeneous graphs. Based on the heterogeneity, unstructured, high dimensionality and temporal characteristics of the log data, a heterogeneous graph is constructed based on the heterogeneous log data. The heterogeneous graph can better reflect the heterogeneity and temporal nature of the log data, and can accurately characterize the log data. Log detection based on heterogeneous graphs and heterogeneous graph models can improve the accuracy of log detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a log detection method based on a heterogeneous graph provided in an embodiment of this specification; Figure 2 It is a processing flow chart of a log detection method based on a heterogeneous graph applied to digital service network log anomaly detection provided in an embodiment of this specification; Figure 3 It is a structural schematic diagram of a log detection device based on a heterogeneous graph provided in an embodiment of this specification; Figure 4 It is a structural block diagram of a computing device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0019] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0020] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0022] First, the terms involved in one or more embodiments of this specification are explained.

[0023] Transformer: A deep learning model based on the self-attention mechanism, which abandons the sequence dependency in traditional recurrent neural networks (RNN) and convolutional neural networks (CNN) and instead uses the attention mechanism to process the information in the input sequence. The core advantage of the Transformer model lies in its parallel computing capability, which makes it more efficient when processing large-scale data sets.

[0024] BERT (Bidirectional Encoder Representations from Transformers): is a natural language processing (NLP) pre-training model based on the Transformer architecture. The innovation of BERT is that it can pre-train deep bidirectional representations by jointly considering the left and right contexts in all layers. This bidirectional pre-training enables BERT to capture richer contextual information when understanding language.

[0025] SemanticFormer: A novel Transformer model that is specifically designed to integrate and exploit semantic information in multidimensional datasets. This model is able to learn discriminative visual representations by exploring the interactions between different semantic markers, and is applicable to a variety of computer vision tasks such as hyperspectral image classification and trajectory prediction in traffic scenes.

[0026] In this specification, a log detection method based on a heterogeneous graph is provided. This specification also relates to a log detection device based on a heterogeneous graph, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0027] Figure 1 A flow chart of a log detection method based on a heterogeneous graph provided according to an embodiment of this specification is shown, which specifically includes the following steps: Step 102: Obtain heterogeneous log data of the target service, and construct at least two heterogeneous graphs containing multiple types of nodes based on the heterogeneous log data.

[0028] Specifically, the target service is a heterogeneous network service, and the target service can be a service provided by a digital service network. Heterogeneous log data is the log data generated in the process of the target service providing services to users. Since the target service has heterogeneous properties, the heterogeneous log data also has heterogeneous properties. The acquisition of heterogeneous log data needs to retain the heterogeneity of the semantics in the log data and the dynamic temporal correlation between the log data. A heterogeneous graph is a graph structure constructed with the components in the heterogeneous log data as nodes and the relationships between the components as edges. A heterogeneous graph is also regarded as a log snapshot. The heterogeneous graph contains different types of nodes, and different types of nodes correspond to the components in the heterogeneous log data.

[0029] Based on this, the heterogeneous log data of the target service is obtained. In the case where the heterogeneous log data is a log sequence, a heterogeneous graph corresponding to the heterogeneous log sequence containing multiple types of nodes is constructed.

[0030] In practical applications, log data is usually dynamic, heterogeneous, and correlated. Dynamic and heterogeneous means that log data is constantly updated over time, and different time periods may contain log information of different types and formats. Correlation means that there is often a connection between log entries, such as different stages of the same event or interactions between different system components. In order to more effectively capture these dynamic heterogeneities and correlations, a heterogeneous log snapshot, i.e., a heterogeneous graph, is constructed based on heterogeneous log data.

[0031] Furthermore, heterogeneous log data are usually associated. When processing heterogeneous log data, the processing can be performed based on the log sequence corresponding to the heterogeneous log data. The specific implementation is as follows: Determine a log sequence corresponding to the heterogeneous log data, and extract at least two target logs from the log sequence; and construct heterogeneous graphs corresponding to the at least two target logs based on a target node corresponding to each target log and a node type of the target node.

[0032] Specifically, a log sequence refers to log data with time correlation in heterogeneous log data, and is obtained by arranging them in chronological order. A target log refers to a log sequence within a period of time intercepted in the log sequence according to the order in which the logs are arranged. The extraction of at least two target logs can adopt a sliding window algorithm to divide the log sequence into a group of log sequences at a fixed time interval, and a group of log sequences is at least two target logs. The target node corresponding to the target log can be a component contained in the log data of the target log, and the node type corresponding to the target log is at least one.

[0033] Based on this, at least two components of each data log in the heterogeneous log data are determined, and each log data corresponds to a timestamp. According to the timestamp in the log data, the log sequence corresponding to the heterogeneous log data can be determined. In the log sequence, a sliding window algorithm is used to extract a set of target log sequences at a fixed time interval. A set of target log sequences contains at least two target logs, and a target log is a log sequence. Based on the target node corresponding to each target log and the node type of the target node, a heterogeneous graph corresponding to at least two target logs is constructed.

[0034] For example, the heterogeneous log data in the digital service network is extracted. The heterogeneous log data contains multiple log data, each of which can contain services, timestamps, templates, and parameters. The heterogeneous log data is used to form a log sequence with a timestamp. The sliding window algorithm is used to divide the log sequence into a set of target log sequences at fixed time intervals. For each target log sequence, the nodes and the edges between the nodes are determined, and the heterogeneous graph corresponding to the target log sequence is constructed.

[0035] In summary, the log sequence corresponding to the heterogeneous log data is determined. After the log sequence is divided, a heterogeneous graph corresponding to each target log is constructed, and the temporal correlation of the target log is reflected in the heterogeneous graph through the connection between the nodes in the heterogeneous graph.

[0036] Furthermore, in order to reflect the dynamic temporal sequence between log data in the target log in the heterogeneous graph, time information can be added when constructing the heterogeneous graph. The specific implementation is as follows: Determine the target node corresponding to the target log and the node type of the target node; determine the target node pair in the target node according to the node type, and determine the target edge of the target node pair; construct an initial heterogeneous graph based on the target node pair and the target edge, and assign time information to the nodes and edges in the initial heterogeneous graph to obtain the heterogeneous graph corresponding to the target log.

[0037] Specifically, the target node is constructed based on the components of the log data in the target log. The target node represents the components of the log data, and the node type refers to the data type corresponding to the components of the log data in the target log. Node types include but are not limited to service nodes, event nodes, and parameter nodes. The target node pair is composed of nodes of two node types with an association relationship. The event node can form a target node pair with its associated service node, and the event node can form a target node pair with its associated parameter node. The edge between the two nodes in the target node pair is constructed and used as the target edge. Time information refers to the time when the event occurs corresponding to the nodes and edges in the initial heterogeneous graph.

[0038] Based on this, determine the target node corresponding to the target log and the node type of the target node. Select two nodes with a type association relationship in the target node according to the node type to form a target node pair, and determine the target edge between the two nodes in the target node pair. Construct an initial heterogeneous graph based on the target node pair and the target edge, and assign time information to the nodes and edges in the initial heterogeneous graph to obtain the heterogeneous graph corresponding to the target log. For the event nodes in the initial heterogeneous graph, the time information can be the time when the log data corresponding to the event node occurs; for the service nodes and parameter nodes, the time information can be the latest occurrence time of the associated log data; for the edges in the initial heterogeneous graph, the time information can be the time information of the associated event nodes.

[0039] Following the above example, for each target log, a heterogeneous graph corresponding to the target log is constructed. The event nodes, service nodes, and parameter nodes contained in the target log are determined. The event nodes are connected to their corresponding service nodes, and the parameter nodes corresponding to the event nodes are connected to form an initial heterogeneous graph. The time of the most recent log is assigned to all heterogeneous nodes and edges associated with it to obtain a heterogeneous graph.

[0040] In summary, an initial heterogeneous graph is constructed based on target node pairs and target edges, and time information is assigned to the nodes and edges in the initial heterogeneous graph respectively to obtain the heterogeneous graph corresponding to the target log, so as to realize the use of heterogeneous graphs to reflect the temporal relationship between different types of nodes, as well as the heterogeneity and dynamics of log data.

[0041] Step 104: inputting at least two heterogeneous graphs into the heterogeneous graph model respectively to obtain semantic alignment information corresponding to the at least two heterogeneous graphs respectively.

[0042] Specifically, after obtaining the heterogeneous log data of the target service as described above and constructing at least two heterogeneous graphs containing multi-type nodes based on the heterogeneous log data, the at least two heterogeneous graphs can be respectively input into the heterogeneous graph model to obtain the semantic alignment information corresponding to the at least two heterogeneous graphs, wherein the heterogeneous graph model can be a pre-trained model, and the heterogeneous graph model provides an encoding function for encoding the heterogeneous graph. The structure of the heterogeneous graph model can be a dual-tower encoder including two Transformer submodules, and the dual towers share a switchable masked self-attention layer to control the interaction between the two towers. Semantic alignment information refers to the information obtained by semantically aligning event nodes and non-event nodes in the heterogeneous graph. Semantic alignment information is obtained by aligning the semantic representation of non-event nodes in the heterogeneous graph to the event representation space of event nodes.

[0043] Based on this, after obtaining the heterogeneous log data of the target service as mentioned above and constructing at least two heterogeneous graphs containing multi-type nodes based on the heterogeneous log data, the at least two heterogeneous graphs are respectively input into the heterogeneous graph model, and the dual-tower encoder in the heterogeneous graph model is used to align the semantic space of the event nodes and non-event nodes in the heterogeneous graph to obtain the semantic alignment information corresponding to the at least two heterogeneous graphs.

[0044] Furthermore, when determining the semantic alignment information of the target heterogeneous graph, it can be achieved by combining the left encoder and the right encoder in the heterogeneous graph model, and the specific implementation is as follows: Determine event nodes and non-event nodes contained in the target heterogeneous graph; process the event nodes based on the left encoder in the heterogeneous graph model to obtain event information, and process the non-event nodes based on the right encoder in the heterogeneous graph model to obtain non-event information; semantically align the event information and the non-event information in the event representation space to obtain semantic alignment information.

[0045] Specifically, event nodes refer to nodes built based on the event component in the target heterogeneous graph, and non-event nodes are service nodes and parameter nodes. The left encoder refers to the left tower encoder contained in the Transformer submodule in the heterogeneous graph model, which is used to encode event nodes, that is, the standard Transformer architecture is used to encode event template text; the right encoder refers to the right tower encoder contained in the Transformer submodule in the heterogeneous graph model, which is used for semantic alignment, learning the original non-event semantic representation from the pre-trained BERT model, and aligning it to the event representation space. Event information is the event representation obtained by encoding event nodes using the left encoder. Non-event information is the non-event representation obtained by encoding non-event nodes by the right encoder. Semantic alignment is achieved using the self-attention mechanism.

[0046] Based on this, the event nodes and non-event nodes contained in the target heterogeneous graph are determined. The left encoder and the right encoder in the heterogeneous graph model are determined. The event nodes are processed based on the left encoder in the heterogeneous graph model to obtain event information, and the non-event nodes are processed based on the right encoder in the heterogeneous graph model to obtain non-event information. The left encoder and the right encoder share a self-attention mechanism to semantically align event information and non-event information in the event representation space, that is, align non-event information to the space of event information to obtain semantic alignment information.

[0047] Continuing with the above example, the encoding module in the heterogeneous graph model is a SemanticFormer structure, which contains a dual-tower encoder with two Transformer submodules. The dual towers share a switchable masked self-attention layer to control the interaction between the two towers, so as to adjust for different pre-training optimization goals. The left-tower encoder is used to encode the event template text. It adopts the standard Transformer architecture and shares the self-attention mechanism with the right tower. It is responsible for encoding the event template text into an event representation. The right-tower encoder is used for semantic alignment and is responsible for learning the original non-event semantic representation from the pre-trained BERT model and aligning it to the event representation space. Each Transformer block of the right-tower encoder is equipped with an aligned cross-attention layer for interacting with the original non-event representation. The heterogeneous graph model can align the features of different nodes to the representation space of the event template.

[0048] In summary, semantic alignment is performed on event information and non-event information in the event representation space to obtain semantic alignment information, and the features of different nodes are aligned to the representation space of the event template, which facilitates subsequent log anomaly detection.

[0049] Step 106: Determine detection information of the heterogeneous log data based on at least two pieces of semantic alignment information and the at least two heterogeneous graphs.

[0050] Specifically, after respectively inputting at least two heterogeneous graphs into the heterogeneous graph model and obtaining the semantic alignment information corresponding to the at least two heterogeneous graphs, the detection information of the heterogeneous log data can be determined based on the at least two semantic alignment information and the at least two heterogeneous graphs, wherein the detection information refers to the detection result of the heterogeneous log data, and the detection information may be abnormal information of the detected heterogeneous log data.

[0051] Based on this, after inputting at least two heterogeneous graphs into the heterogeneous graph model respectively to obtain the semantic alignment information corresponding to the at least two heterogeneous graphs, anomaly detection is performed on the heterogeneous log data based on the at least two semantic alignment information and the at least two heterogeneous graphs to obtain the detection information of the heterogeneous log data.

[0052] Furthermore, when detecting at least two heterogeneous graphs, the abnormal nodes can be first determined in the heterogeneous graphs, the abnormal heterogeneous graphs can be marked, and then the abnormal heterogeneous graphs can be detected. The specific implementation is as follows: Based on at least two semantic alignment information, time dependency information corresponding to each heterogeneous graph is generated respectively, and based on at least two time dependency information and the at least two heterogeneous graphs, an abnormal heterogeneous graph containing abnormal nodes is determined; based on the abnormal nodes contained in the abnormal heterogeneous graph, detection information of the heterogeneous log data is determined.

[0053] Specifically, the time dependency information refers to the time dependency information of the heterogeneous graph determined by using the message passing architecture, which can be used to detect abnormal situations caused by malicious associations in log data in addition to detecting common anomalies in log data. The abnormal heterogeneous graph can be determined based on a single classification method, and the abnormal heterogeneous graph containing abnormal nodes is determined based on the distribution corresponding to at least two heterogeneous graphs.

[0054] Based on this, the time dependency information corresponding to each heterogeneous graph is generated based on at least two semantic alignment information. A heterogeneous graph distribution is constructed based on at least two time dependency information and at least two heterogeneous graphs, and nodes with a longer distance in the heterogeneous graph distribution are determined as abnormal nodes, and the heterogeneous graphs corresponding to the abnormal nodes are used as abnormal heterogeneous graphs. The detection information of the heterogeneous log data is determined based on the abnormal nodes contained in the abnormal heterogeneous graph.

[0055] In summary, an abnormal heterogeneous graph containing abnormal nodes is determined based on at least two time-dependent information and at least two heterogeneous graphs, and detection information of heterogeneous log data is determined based on the abnormal nodes contained in the abnormal heterogeneous graph. Heterogeneous information in the heterogeneous graph is obtained, and anomaly detection is performed based on the heterogeneous information to improve the accuracy of detection.

[0056] Furthermore, when performing anomaly detection on at least two heterogeneous graphs, a combination of intra-heterogeneous graph encoding and inter-heterogeneous graph encoding may be used, which is specifically implemented as follows: Based on the at least two semantic alignment information, the at least two heterogeneous graphs are encoded within the heterogeneous graph to obtain heterogeneous information corresponding to each heterogeneous graph; and information extraction is performed on the heterogeneous information corresponding to each heterogeneous graph to obtain time dependency information corresponding to each heterogeneous graph.

[0057] Specifically, encoding in heterogeneous graphs includes two steps: message passing and message aggregation. The heterogeneous information of the heterogeneous graph can be obtained through message passing, and the time-dependent information can be obtained by message aggregation based on the heterogeneous information.

[0058] Based on this, at least two heterogeneous graphs are encoded within the heterogeneous graph based on at least two semantic alignment information, and at least two heterogeneous graphs are encoded within the graph by message passing and message aggregation in sequence to obtain heterogeneous information corresponding to each heterogeneous graph. The heterogeneous information corresponding to each heterogeneous graph is extracted by inter-heterogeneous graph encoding to obtain time-dependent information corresponding to each heterogeneous graph.

[0059] Using the above example, considering that the time interval between logs plays an important role in log anomaly detection, when encoding heterogeneous graphs, at least two heterogeneous graphs are processed by intra-heterogeneous graph encoding and inter-heterogeneous graph encoding. The heterogeneous graphs are encoded intra-heterogeneously by message passing and message aggregation to obtain heterogeneous information of the heterogeneous graphs. The heterogeneous information is input into the Transformer to implement inter-heterogeneous graph encoding to obtain the time dependency information of at least two heterogeneous graphs.

[0060] In summary, by combining intra-heterogeneous graph coding with inter-heterogeneous graph coding, the time dependency information of at least two heterogeneous graphs is obtained, thereby improving the comprehensiveness of subsequent anomaly detection.

[0061] Furthermore, when performing anomaly detection on at least two heterogeneous graphs, a single classification scoring mechanism can be used, which is specifically implemented as follows: Determine a preset detection node, and respectively determine the nodes to be detected in the at least two heterogeneous graphs; perform distance detection on the nodes to be detected corresponding to the at least two heterogeneous graphs respectively based on at least two time-dependent information and the preset detection node, and determine the abnormal node according to the detection result and the distance threshold.

[0062] Specifically, the preset detection node can be a central node for constructing node distribution. The node to be detected refers to a node contained in a heterogeneous graph. Distance detection refers to detecting the distance between the node to be detected and the preset detection node. The distance threshold is used to distinguish abnormal nodes. When the distance obtained by the distance detection is greater than the distance threshold, the detected node to be detected is considered to be an abnormal node.

[0063] Based on this, a preset detection node is determined, and the nodes to be detected are determined in at least two heterogeneous graphs. A node distribution graph is constructed based on the nodes to be detected, at least two time-dependent information and the preset detection nodes, and the distance between each node to be detected and the preset detection node is calculated in the node distribution graph, that is, the distance detection is performed on the nodes to be detected respectively, and the distance is scored. When the distance score of the detection result is greater than the distance threshold, the node to be detected is determined to be an abnormal node; when the distance score of the detection result is less than the distance threshold, the node to be detected is determined to be a non-abnormal node.

[0064] In summary, based on at least two time-dependent information and preset detection nodes, distance detection is performed on the nodes to be detected corresponding to at least two heterogeneous graphs respectively, and abnormal nodes are determined according to the detection results and the distance threshold, thereby improving the simplicity of abnormal node judgment.

[0065] Furthermore, the pre-training of the heterogeneous graph model includes: Determine a sample heterogeneous graph, input the sample heterogeneous graph into an initial heterogeneous graph model, and determine the node alignment information of the sample heterogeneous graph through global alignment matching and local alignment matching; determine the sample label of the sample heterogeneous graph, and adjust the parameters of the initial heterogeneous graph model based on the sample label, the node alignment information and the loss function until the heterogeneous graph model that meets the pre-training stop condition is obtained.

[0066] Specifically, the sample heterogeneous graph refers to a sample used to train the heterogeneous graph model. The initial heterogeneous graph model is an untrained heterogeneous graph model. In the global alignment matching process, for the sample heterogeneous graph, non-events and events are determined, and event representations and non-event representations are obtained from the left tower encoder and the right tower encoder of the initial heterogeneous graph model, respectively, and the pairwise similarity between the two is calculated. By comparing the similarity of the positive sample pair with the similarity of the negative sample pair, the non-event representation is aligned to the same space of the event representation. In the process of local alignment matching, it is predicted whether a pair of non-events and events match, so as to further learn the fine-grained alignment between the non-event representation and the event representation at the local level. The node alignment information is the prediction result of the initial heterogeneous graph model for the sample heterogeneous graph. The loss function can use the cross entropy loss function. The sample label is the correct anomaly detection result of the sample heterogeneous graph. The pre-training stop condition can be that the prediction accuracy of the heterogeneous graph model reaches a preset accuracy threshold, or that a preset model training round is completed, or that a preset training duration is reached. This embodiment does not impose any limitation on this.

[0067] Based on this, determine the sample heterogeneous graph. Input the sample heterogeneous graph into the initial heterogeneous graph model, and determine the node alignment information of the sample heterogeneous graph through global alignment matching and local alignment matching. Determine the sample label of the sample heterogeneous graph, and adjust the parameters of the initial heterogeneous graph model based on the sample label, node alignment information and loss function to obtain the intermediate heterogeneous graph model. Determine the next sample heterogeneous graph and input it into the intermediate heterogeneous graph model for prediction, and so on, until a heterogeneous graph model that meets the pre-training stop condition is obtained.

[0068] Using the above example, the initial heterogeneous graph model includes an initial left tower encoder and an initial right tower encoder. The event nodes in the sample heterogeneous graph are input into the initial left tower encoder, and the non-event nodes are input into the initial right tower encoder for prediction. Then, the event representation output by the initial left tower encoder and the non-event representation output by the right tower encoder are aligned to obtain node alignment information. The loss value is calculated by combining the cross entropy loss function and the sample label, and the parameters of the initial heterogeneous graph model are adjusted. This process is repeated until a heterogeneous graph model that meets the training stop condition is obtained.

[0069] In summary, the parameters of the initial heterogeneous graph model are adjusted based on sample labels, node alignment information and loss function until a heterogeneous graph model that meets the pre-training stop conditions is obtained. A heterogeneous graph model with strong anomaly detection capabilities is trained to improve the accuracy of anomaly detection for heterogeneous log data.

[0070] The log detection method based on heterogeneous graphs provided in this specification obtains heterogeneous log data of the target service, and constructs at least two heterogeneous graphs containing multi-type nodes based on the heterogeneous log data. The at least two heterogeneous graphs are respectively input into the heterogeneous graph model to obtain the semantic alignment information corresponding to the at least two heterogeneous graphs. The detection information of the heterogeneous log data is determined based on the at least two semantic alignment information and the at least two heterogeneous graphs. Based on the heterogeneity, unstructured, high dimensionality and temporal characteristics of the log data, a heterogeneous graph is constructed based on the heterogeneous log data. The heterogeneous graph can better reflect the heterogeneity and temporal nature of the log data, and can accurately characterize the log data. Log detection based on heterogeneous graphs and heterogeneous graph models can improve the accuracy of log detection.

[0071] The following combination Figure 2 Taking the application of the heterogeneous graph-based log detection method provided in this specification in the digital service network log anomaly detection as an example, the heterogeneous graph-based log detection method is further described. Figure 2 A processing flow chart of a log detection method based on a heterogeneous graph for detecting anomalies in a digital service network log provided in an embodiment of the present specification is shown, which specifically includes the following steps: Step 202: Obtain heterogeneous log data, and divide the heterogeneous log data into a group of log sequences.

[0072] In the digital service network scenario, anomaly detection can be performed on the logs generated during the digital network service process. The log data generated in the digital service network scenario is usually dynamic, heterogeneous, and correlated. Obtain heterogeneous log data and parse it. Determine the various components of each log data, including service, timestamp, template, and parameters.

[0073] The service can be determined by the file associated with the log data, the service can be 'Apache'; the timestamp, template and parameters are determined from the log data. A template can be 'CE sym〈*〉,at〈*〉,mask〈*〉', with a set of parameters, ['2','0x0b85f680','0x0e'], and the corresponding timestamp '29 / Feb / 2020:07:24:47+0000'. After obtaining the log sequence with timestamps, a sliding window method is used to Divide the log sequence into a set of log sequences ,in represents the i-th log sequence, Represents the jth log in the sequence.

[0074] Step 204: determine the event nodes and non-event nodes of each log sequence in a set of log sequences, and determine the relationship between the nodes.

[0075] For each log sequence, a corresponding heterogeneous graph is constructed to obtain a heterogeneous dynamic graph. The heterogeneous graph contains three types of nodes and two types of edges. The nodes include: service nodes, which represent a service by the service name; event nodes, which represent a separate log by the template text; parameter nodes, which represent a parameter by the parameter value corresponding to the template. Parameter nodes and service nodes are non-event nodes. The two types of edges include connecting event nodes and their corresponding service nodes, and connecting parameter nodes corresponding to event nodes.

[0076] Step 206: construct an initial heterogeneous graph based on the event nodes, non-event nodes and relationships between nodes of each log sequence in a set of log sequences.

[0077] The initial heterogeneous graph is constructed based on three types of nodes and two types of edges.

[0078] Step 208: Configure time attributes for the edges and nodes included in each initial heterogeneous graph to obtain a heterogeneous graph.

[0079] After the initial heterogeneous graph is constructed, the time of the most recent log is assigned to the nodes and edges in the heterogeneous graph associated with it. For event nodes, the time attribute is set to the occurrence time of the corresponding log; for service nodes and parameter nodes, the time attribute is set to the latest occurrence time of the associated log; for various types of edges, the time attribute is set to the time attribute of the associated event node.

[0080] Step 210: Encode each heterogeneous graph using the left encoder and the right encoder in the heterogeneous graph model to obtain semantic alignment information.

[0081] The heterogeneous graph model is a SemanticFormer structure, which is a dual-tower encoder containing two Transformer submodules. The dual towers share a switchable masked self-attention layer to control the interaction between the two towers, so as to adjust for different pre-training optimization goals. The left-tower encoder is used to encode the event template text. It adopts the standard Transformer architecture and shares the self-attention mechanism with the right tower. It is responsible for encoding the event template text into event representation. The right-tower encoder is used for semantic alignment and is responsible for learning the original non-event semantic representation from the pre-trained BERT model and aligning it to the event representation space. Each Transformer block of the right-tower encoder is equipped with an aligned cross-attention layer for interacting with the original non-event representation. By encoding each heterogeneous graph using the left encoder and the right encoder in the heterogeneous graph model, the semantic alignment information of the heterogeneous graph can be obtained.

[0082] The specific calculation process of the cross attention layer is as follows: (1) in, , Q (Query) represents the query vector, which is a vector representing the input token (for example, a word) currently being processed; K (Key) represents the key vector, which is the representation vector of all other tokens in the input sequence; V (Value) represents the value vector, which is the representation vector of the information or content associated with each key. represents the non-event semantic representation learned from the pre-trained BERT model, represents the query embedding of the science department, represents the transpose of the key vector K, and h represents the output of the cross-attention layer.

[0083] The main purpose of the cross-attention layer is to determine which input tokens are most important for understanding the current token by comparing the query vector and the key vector. This is usually done by calculating the similarity score between Q and K, and then using the softmax function to convert these scores into a normalized probability distribution. This distribution is called the attention weight, which represents the relative importance of each token in the sequence to the current token. Using these attention weights, the value vector is weighted and summed to obtain a new vector that comprehensively considers the information of the entire input sequence, that is, the node feature vector that completes the semantic alignment.

[0084] In actual applications, for each event node, the event template text is input into the left encoder to obtain the embedding of the [CLS] tag as the representation of the event node; for each service node and parameter node, the original service name and original parameter value are input into the right encoder to obtain the aligned non-event representation as the embedding of the node. In this way, the representation of semantic nodes can be obtained.

[0085] Pre-training SemanticFormer enables the non-event representations aligned by the right-tower encoder and the event representations learned by the left-tower encoder to be unified in the same semantic representation space. Different attention mask strategies are adopted in the self-attention layer, sharing the same input format and model parameters, and jointly optimizing the pre-training goals of global and local alignment. The details are as follows: Globally aligned contrastive learning: A unidirectional self-attention mask is used to avoid information leakage, and all queries and events cannot interact with each other. For each pair of non-events and events in the batch training data, the event representation and non-event representation are obtained from the left tower and right tower encoders respectively, and the pairwise similarity between the two is calculated. By comparing the similarity of the positive sample pairs with the similarity of the negative sample pairs, the non-event representation is aligned to the same space of the event representation.

[0086] Local alignment matching: Perform a binary classification task to predict whether a pair of non-events and events matches, so as to further learn fine-grained alignment between non-event representations and event representations at the local level. Using a bidirectional self-attention mask, queries and events can interact; the non-event representation obtained from the right tower is regarded as the fused representation of the pair of representations, and is input into a binary linear classifier, and then the probability of matching is predicted through softmax. A positive and negative sample strategy is adopted to select unmatched negative sample pairs that are semantically similar but different in details. Specifically, for each event, samples are sampled from unconnected services, and parameters that are unrelated to the log corresponding to the event are sampled, which appear in other logs but belong to the same service related to the log. All sampling operations are based on pairwise similarity distributions. The closer the non-event is to the event, the higher the chance of it being sampled.

[0087] Step 212: Perform intra-heterogeneous graph encoding and inter-heterogeneous graph encoding based on the semantic alignment information and the heterogeneous graph to obtain event representation of the heterogeneous graph.

[0088] After obtaining log sequences with rich semantic representations, in order to improve the effect of log anomaly detection, it is necessary to learn log representations based on the intra-graph relations and inter-graph relations of heterogeneous graphs. It is also necessary to consider the inherent category imbalance problem in the log anomaly detection task. Therefore, intra-graph encoding and inter-graph encoding are adopted. Intra-graph encoding includes message passing and message aggregation.

[0089] The message transmission process is as follows: The time interval between logs plays an important role in log anomaly detection, so based on the node pair The relative time interval between them additionally encodes the temporal embedding of the edge during the message passing process. Edges on layers , from the node To Node Calculate the multi-head attention message as follows:

[0090] in, represents the feature embedding representation of the node at the kth layer, Indicates the node type of node v, Represents the linear mapping layer of the current node type at layer i, Indicates the multi-head attention message, represents the time embedding vector calculated based on Fourier series, represents the temporal embedding vector at the k-1th layer.

[0091] The message aggregation process is as follows: When aggregating messages on heterogeneous graphs, an attention mechanism is used , used to calculate each source node The importance of the node is then aggregated from all neighbor nodes to the target node. , the calculation formula is as follows (3):

[0092] in, represents the set of all neighbor nodes of node v, Represents the embedded representation obtained at the k-th layer node v.

[0093] When encoding between heterogeneous graphs, the node embeddings generated by the intra-heterogeneous graph encoder capture the spatial information of the logs. However, the intra-heterogeneous graph encoder fails to fully consider the temporal dynamics inherent in the logs, which is still insufficient for detecting anomalies caused by malicious associations. Therefore, the node embedding sequence from the intra-heterogeneous graph encoder is input into the Transformer to extract the temporal dependency information.

[0094] Step 214: Combined with the event representation of the heterogeneous graph, a single-classification scoring mechanism is used to perform anomaly detection on the heterogeneous graph to obtain abnormal nodes in the heterogeneous graph.

[0095] A single-class scoring mechanism is used to detect anomalies in heterogeneous graphs. The single-class scoring mechanism is used to learn normal patterns and mark deviations from normal patterns as anomalies. Set a central position To represent the typical distribution of normal patterns, the distance between the event node and the center position is calculated to obtain the anomaly score. Accordingly, a threshold is set to distinguish normal and abnormal nodes. When the anomaly score of a node is greater than the threshold, the node is predicted to be an abnormal node; otherwise, the node is predicted to be a normal node.

[0096] In addition, this embodiment also involves pre-training of SemanticFormer: by minimizing To pre-train SemanticFormer, where represents the global alignment contrast learning loss, represents the local alignment matching loss. The global alignment contrast learning loss is based on the softmax cross entropy loss, as shown in the following formula (4):

[0097] in, represents a positive sample non-event node, Represents the embedding representation of the corresponding nodes, such as service nodes and parameter nodes; Represents an event node, represents the embedding representation of the corresponding node; represents a set of non-event nodes containing positive samples and sampled negative samples, represents the similarity score between two representations.

[0098] The local alignment loss is based on the cross entropy loss, as shown in the following formula (5):

[0099] in, represents the constructed matching training dataset, Represents a non-event node With event nodes The probability of whether the matching is connected in the log graph, The actual label indicating whether two nodes match.

[0100] You can also fine-tune abnormal log detection: by minimizing To fine-tune the downstream abnormal log detection task. As shown in the following formula (6):

[0101] in, represents the embedding of the target node, Indicates the center position of normal logging mode.

[0102] In summary, the abnormal log detection of digital service network is carried out by introducing heterogeneous dynamic graphs. Heterogeneous dynamic graphs can better express the heterogeneity and temporality of log content, which is a more accurate way to describe logs and improve the accuracy of anomaly detection of abnormal logs.

[0103] Corresponding to the above method embodiment, this specification also provides a log detection device embodiment based on a heterogeneous graph. Figure 3 FIG. 1 shows a schematic diagram of a log detection device based on a heterogeneous graph provided in an embodiment of the present specification. Figure 3 As shown, the device comprises: An acquisition module 302 is configured to acquire heterogeneous log data of a target service and construct at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data; An input module 304 is configured to input at least two heterogeneous graphs into the heterogeneous graph model respectively, and obtain semantic alignment information corresponding to the at least two heterogeneous graphs respectively; The determination module 306 is configured to determine the detection information of the heterogeneous log data based on at least two pieces of semantic alignment information and the at least two heterogeneous graphs.

[0104] In an optional embodiment, the acquisition module 302 is further configured to: Determine a log sequence corresponding to the heterogeneous log data, and extract at least two target logs from the log sequence; Based on the target node corresponding to each target log and the node type of the target node, heterogeneous graphs corresponding to the at least two target logs are constructed.

[0105] In an optional embodiment, the acquisition module 302 is further configured to: Determine a target node corresponding to the target log and a node type of the target node; Determine a target node pair in the target node according to the node type, and determine a target edge of the target node pair; An initial heterogeneous graph is constructed based on the target node pair and the target edge, and time information is allocated to the nodes and edges in the initial heterogeneous graph to obtain a heterogeneous graph corresponding to the target log.

[0106] In an optional embodiment, the input module 304 is further configured to: Determine event nodes and non-event nodes included in the target heterogeneous graph; Processing the event node based on the left encoder in the heterogeneous graph model to obtain event information, and processing the non-event node based on the right encoder in the heterogeneous graph model to obtain non-event information; Semantically align the event information and the non-event information in an event representation space to obtain semantic alignment information.

[0107] In an optional embodiment, the determining module 306 is further configured to: Generate time dependency information corresponding to each heterogeneous graph based on at least two pieces of semantic alignment information, and determine an abnormal heterogeneous graph containing an abnormal node based on the at least two pieces of time dependency information and the at least two heterogeneous graphs; Detection information of the heterogeneous log data is determined based on the abnormal nodes included in the abnormal heterogeneous graph.

[0108] In an optional embodiment, the determining module 306 is further configured to: Determine a preset detection node, and respectively determine nodes to be detected in the at least two heterogeneous graphs; Based on at least two time-dependent information and the preset detection nodes, distance detection is performed on the nodes to be detected respectively corresponding to the at least two heterogeneous graphs, and the abnormal node is determined according to the detection result and the distance threshold.

[0109] In an optional embodiment, the determining module 306 is further configured to: Performing intra-heterogeneous-graph encoding on the at least two heterogeneous graphs based on the at least two semantic alignment information to obtain heterogeneous information corresponding to each heterogeneous graph; The heterogeneous information corresponding to each heterogeneous graph is extracted to obtain the time-dependent information corresponding to each heterogeneous graph.

[0110] In an optional embodiment, the determining module 306 is further configured to: Determine a sample heterogeneous graph, input the sample heterogeneous graph into an initial heterogeneous graph model, and determine node alignment information of the sample heterogeneous graph through global alignment matching and local alignment matching; Determine a sample label of the sample heterogeneous graph, and adjust the parameters of the initial heterogeneous graph model based on the sample label, the node alignment information and the loss function until the heterogeneous graph model that meets the pre-training stop condition is obtained.

[0111] The log detection device based on heterogeneous graph provided in this specification obtains heterogeneous log data of the target service, and constructs at least two heterogeneous graphs containing multi-type nodes based on the heterogeneous log data. The at least two heterogeneous graphs are respectively input into the heterogeneous graph model to obtain the semantic alignment information corresponding to the at least two heterogeneous graphs. The detection information of the heterogeneous log data is determined based on the at least two semantic alignment information and the at least two heterogeneous graphs. Based on the heterogeneity, unstructured, high dimensionality and temporal characteristics of the log data, a heterogeneous graph is constructed based on the heterogeneous log data. The heterogeneous graph can better reflect the heterogeneity and temporal nature of the log data, and can accurately characterize the log data. Log detection based on heterogeneous graphs and heterogeneous graph models can improve the accuracy of log detection.

[0112] The above is a schematic scheme of a log detection device based on a heterogeneous graph of this embodiment. It should be noted that the technical scheme of the log detection device based on a heterogeneous graph and the technical scheme of the log detection method based on a heterogeneous graph belong to the same concept, and the details of the technical scheme of the log detection device based on a heterogeneous graph that are not described in detail can all be referred to the description of the technical scheme of the log detection method based on a heterogeneous graph.

[0113] Figure 4 The structure block diagram of a computing device 400 provided according to an embodiment of the present specification is shown. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0114] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0115] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0116] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 400 may also be a mobile or stationary server.

[0117] Among them, when the processor 420 executes the computer program or instruction, the steps of the log detection method based on the heterogeneous graph are implemented.

[0118] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the log detection method based on heterogeneous graphs belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the log detection method based on heterogeneous graphs.

[0119] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program or instruction, which implements the steps of the heterogeneous graph-based log detection method as described above when the computer program or instruction is executed by a processor.

[0120] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the log detection method based on heterogeneous graphs belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the log detection method based on heterogeneous graphs.

[0121] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which implements the steps of the above-mentioned heterogeneous graph-based log detection method when executed by a processor.

[0122] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the log detection method based on heterogeneous graphs belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be found in the description of the technical scheme of the log detection method based on heterogeneous graphs.

[0123] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0124] The computer program or instruction includes a computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0125] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0126] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A log detection method based on heterogeneous graph, characterized in that: include: Obtaining heterogeneous log data of a target service, and constructing at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data; Inputting at least two heterogeneous graphs into the heterogeneous graph model respectively to obtain semantic alignment information corresponding to the at least two heterogeneous graphs respectively; Detection information of the heterogeneous log data is determined based on at least two pieces of semantic alignment information and the at least two heterogeneous graphs.

2. The log detection method based on heterogeneous graph according to claim 1, characterized in that: The step of constructing at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data includes: Determine a log sequence corresponding to the heterogeneous log data, and extract at least two target logs from the log sequence; Based on the target node corresponding to each target log and the node type of the target node, heterogeneous graphs corresponding to the at least two target logs are constructed.

3. The log detection method based on heterogeneous graph according to claim 2 is characterized in that: The construction of a heterogeneous graph corresponding to any target log includes: Determine a target node corresponding to the target log and a node type of the target node; Determine a target node pair in the target node according to the node type, and determine a target edge of the target node pair; An initial heterogeneous graph is constructed based on the target node pair and the target edge, and time information is allocated to the nodes and edges in the initial heterogeneous graph to obtain a heterogeneous graph corresponding to the target log.

4. The log detection method based on heterogeneous graph according to claim 1, characterized in that: Determining the semantic alignment information of the target heterogeneous graph in the at least two heterogeneous graphs includes: Determine event nodes and non-event nodes included in the target heterogeneous graph; Processing the event node based on the left encoder in the heterogeneous graph model to obtain event information, and processing the non-event node based on the right encoder in the heterogeneous graph model to obtain non-event information; Semantically align the event information and the non-event information in an event representation space to obtain semantic alignment information.

5. The log detection method based on heterogeneous graph according to claim 1, characterized in that: The determining the detection information of the heterogeneous log data based on the at least two pieces of semantic alignment information and the at least two heterogeneous graphs includes: Generate time dependency information corresponding to each heterogeneous graph based on at least two pieces of semantic alignment information, and determine an abnormal heterogeneous graph containing an abnormal node based on the at least two pieces of time dependency information and the at least two heterogeneous graphs; Detection information of the heterogeneous log data is determined based on the abnormal nodes included in the abnormal heterogeneous graph.

6. The log detection method based on heterogeneous graph according to claim 5 is characterized in that: The determination of the abnormal node includes: Determine a preset detection node, and respectively determine nodes to be detected in the at least two heterogeneous graphs; Based on at least two time-dependent information and the preset detection nodes, distance detection is performed on the nodes to be detected respectively corresponding to the at least two heterogeneous graphs, and the abnormal node is determined according to the detection result and the distance threshold.

7. The log detection method based on heterogeneous graph according to claim 5 is characterized in that: The generating the time dependency information corresponding to each heterogeneous graph based on at least two pieces of semantic alignment information respectively includes: Performing intra-heterogeneous-graph encoding on the at least two heterogeneous graphs based on the at least two semantic alignment information to obtain heterogeneous information corresponding to each heterogeneous graph; The heterogeneous information corresponding to each heterogeneous graph is extracted to obtain the time-dependent information corresponding to each heterogeneous graph.

8. The log detection method based on heterogeneous graph according to claim 1, characterized in that: The pre-training of the heterogeneous graph model includes: Determine a sample heterogeneous graph, input the sample heterogeneous graph into an initial heterogeneous graph model, and determine node alignment information of the sample heterogeneous graph through global alignment matching and local alignment matching; Determine a sample label of the sample heterogeneous graph, and adjust the parameters of the initial heterogeneous graph model based on the sample label, the node alignment information and the loss function until the heterogeneous graph model that meets the pre-training stop condition is obtained.

9. A log detection device based on heterogeneous graph, characterized in that: include: An acquisition module is configured to acquire heterogeneous log data of a target service and construct at least two heterogeneous graphs including nodes of multiple types based on the heterogeneous log data; An input module is configured to input at least two heterogeneous graphs into the heterogeneous graph model respectively, and obtain semantic alignment information corresponding to the at least two heterogeneous graphs respectively; A determination module is configured to determine detection information of the heterogeneous log data based on at least two pieces of semantic alignment information and the at least two heterogeneous graphs.

10. A computing device comprising a memory, a processor, and a computer program or instruction stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program or instructions, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images