Traffic anomaly detection method and device, computer equipment and storage medium

Abstract

The invention provides a traffic anomaly detection method and device, computer equipment and a storage medium. The method comprises the following steps: generating a dynamic heterogeneous graph of a target communication network; performing coding processing on the dynamic heterogeneous graph by using a pre-trained coding model to obtain node coding features corresponding to each node in the dynamic heterogeneous graph; based on normal behavior prototypes of a plurality of normal communication behaviors learned by the neural memory network, addressing the node coding features corresponding to each node to obtain attention weights corresponding to each node and different normal behavior prototypes, and according to the attention weights, selecting the normal behavior prototypes of the plurality of normal communication behaviors according to the attention weights; performing reconstruction processing on the node coding feature corresponding to each node to obtain a node reconstruction feature corresponding to each node; and determining a traffic anomaly detection result corresponding to each node based on the difference information between the node reconstruction features and the node coding features and according to the attention dispersion degree corresponding to the node and determined based on the attention weight.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to an anomaly detection method and device for traffic, a computer device, and a storage medium. BACKGROUND

[0002] Network security is a technology, management and compliance system for protecting network systems, devices, data and services from unauthorized access, tampering, destruction or leakage, and ensuring continuous and reliable operation of the system and uninterrupted network services. The Secure Sockets Layer (SSL) protocol and the Transport Layer Security (TLS) protocol are both encryption protocols for ensuring the security of network communication, and their core functions are to provide encryption, integrity verification and identity authentication for the transmission of data between clients and servers.

[0003] In the current network security environment, the widespread use of encrypted traffic has brought great challenges to traditional security detection methods. Although TLS and SSL encryption protocols protect the privacy of data transmission, they also provide a hidden channel for various network attacks, such as Command and Control (C2) communication, data leakage, lateral movement, and other advanced threats. Traditional deep packet inspection-based methods are completely ineffective when faced with encrypted traffic, while detection methods based on traffic statistical features are difficult to capture complex attack patterns. Further, a method capable of detecting anomalies in traffic in complex network relationships has become a pressing problem. SUMMARY

[0004] Therefore, the present disclosure provides an anomaly detection method and device for traffic, a computer device, and a storage medium.

[0005] Specifically, the present disclosure is implemented by the following technical solutions: In a first aspect, the present disclosure provides an anomaly detection method for traffic, which comprises: generating a dynamic heterogeneous graph of a target communication network; the dynamic heterogeneous graph comprises nodes corresponding to the plurality of target network entities respectively, edges for describing the communication relationship between each node and the corresponding neighbor node, and traffic features corresponding to each edge; using a pre-trained encoding model to perform encoding processing on the dynamic heterogeneous graph to obtain node encoding features corresponding to each node in the dynamic heterogeneous graph respectively; Based on the normal behavior prototypes of various normal communication behaviors learned by the neural memory network, the addressing operation is performed on the node encoding features corresponding to each node to obtain the attention weights corresponding to each node and different normal behavior prototypes. Based on the attention weights, the node encoding features corresponding to each node are reconstructed to obtain the node reconstruction features corresponding to each node. Based on the differences between the node reconstruction features and the node encoding features, and according to the attention dispersion determined based on the attention weight and the node, the traffic anomaly detection result corresponding to each node is determined.

[0006] Optionally, generating the dynamic heterogeneous graph of the target communication network includes: In each of the multiple cycles, traffic data of the target communication network in each cycle is acquired; the traffic data includes: first traffic data of communication between each network entity in the target communication network and other network entities outside the target communication network, and / or second traffic data of communication between each network entity in the target communication network; Based on the traffic data of the target communication network in each period, multiple target network entities are identified, and a dynamic heterogeneous graph based on the mutual communication between the target network entities is generated.

[0007] Optionally, the step of determining multiple target network entities based on the traffic data of the target communication network in each period, and generating a dynamic heterogeneous graph based on the communication between the target network entities, includes: Based on the traffic data for each period, traffic metadata for each period is determined; the traffic metadata includes at least one of the following: traffic identification information, traffic time-series data, traffic statistics data, domain name and Internet Protocol (IP) address pairs, and transport layer security protocol certificates. Based on the traffic metadata of each period, multiple target network entities in each period are identified, and the communication relationships between different target network entities are determined, generating traffic time-series characteristics and traffic statistics characteristics between every two target network entities with communication relationships. Nodes corresponding to the multiple target network entities in each cycle are generated, and edges between different target network entities are generated according to the communication relationship between them. The traffic time sequence characteristics and traffic statistics characteristics between every two target network entities with communication relationship are used to generate the communication traffic characteristics of the corresponding edge.

[0008] Optionally, the encoding model includes: an edge attribute encoder and a node encoder; The process of encoding the dynamic heterogeneous graph using a pre-trained encoding model to obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph includes: The flow characteristics of the edges of each node are encoded using the edge attribute encoder to obtain the flow encoding characteristics of the edges corresponding to each node. The node encoder is used to encode each node to obtain the node encoding features corresponding to each node; wherein, the node encoding features corresponding to each node are obtained by feature aggregation of the stream encoding features of each node, the neighboring nodes of each node, and the edges of each node.

[0009] Optionally, the communication traffic characteristics of each edge corresponding to each node include time-series traffic characteristics and statistical traffic characteristics; The step of encoding the flow features of the edges of each node using the edge attribute encoder to obtain the flow coding features of the edges corresponding to each node includes: The temporal flow features corresponding to each edge of each node are subjected to a first encoding process to obtain the first stream encoding features corresponding to each edge; as well as, The statistical flow characteristics of each edge corresponding to each node are subjected to a second encoding process to obtain the second flow encoding characteristics corresponding to each edge; The first stream coding feature and the second stream coding feature corresponding to each edge are concatenated to obtain the concatenated coding feature. The concatenated coding feature is then subjected to global nonlinear mapping and dimensional transformation to obtain the stream coding feature corresponding to each edge.

[0010] Optionally, the time-series traffic features include: time-series traffic features corresponding to multiple time windows respectively; The first encoding process, which performs a first encoding process on the temporal flow features of each edge corresponding to each node to obtain the first stream encoding features corresponding to each edge, includes: For each edge corresponding to each node, based on the self-attention mechanism, the temporal flow features corresponding to multiple time windows are encoded to obtain the temporal encoding features corresponding to each edge in multiple time windows. The temporal coding features corresponding to each edge in each time window and the auxiliary statistical features of the corresponding time window are concatenated to obtain the enhanced temporal coding features of each edge in each time window. According to the order of time windows, the enhanced temporal coding features corresponding to each edge in multiple time windows are combined to form a temporal coding feature sequence, and the temporal coding feature sequence is convolved to obtain the first stream coding feature corresponding to each edge.

[0011] Optionally, the second encoding process for the statistical flow features of each edge corresponding to each node to obtain the second flow encoding features corresponding to each edge includes: The statistical flow features of each edge corresponding to each node are nonlinearly mapped and dimensionally transformed using a multilayer perceptron to obtain the second flow coding features corresponding to each edge.

[0012] Optionally, determining the traffic anomaly detection result for each node based on the difference information between the node reconstruction features and the node encoding features, and according to the attention dispersion determined based on the attention weight and corresponding to the node, includes: Based on the differences between the node reconstruction features and the node encoding features, a reconstruction error score is determined; and an attention dispersion score is determined based on the attention weights corresponding to each node and different normal behavior prototypes. Based on the reconstruction error score and the attention distraction score, the normal behavior prototype group to which each node belongs is determined; The reconstruction error score and the attention distraction score are compared with the baseline threshold corresponding to the normal behavior prototype group to which each node belongs; If either the reconstruction error score or the attention dispersion score corresponding to any node is greater than the corresponding baseline threshold, the node is determined to be an abnormal node.

[0013] Optionally, the method further includes: for abnormal nodes, based on the difference information corresponding to each edge corresponding to the abnormal node, using a gradient backpropagation mechanism to determine the abnormal contribution degree of each edge corresponding to the abnormal node; The target abnormal edge is determined based on the abnormal contribution of each edge corresponding to the abnormal node.

[0014] Secondly, embodiments of this disclosure also provide an anomaly detection device for traffic flow, comprising: A generation module is used to generate a dynamic heterogeneous graph of the target communication network; the dynamic heterogeneous graph includes: nodes corresponding to the plurality of target network entities, edges describing the communication relationship between each node and its corresponding neighboring nodes, and traffic features corresponding to each edge; The encoding module is used to encode the dynamic heterogeneous graph using a pre-trained encoding model to obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph. The reconstruction module is used to perform addressing operations on the node encoding features corresponding to each node based on normal behavior prototypes of various normal communication behaviors learned by the neural memory network, to obtain the attention weights corresponding to each node and different normal behavior prototypes respectively, and to reconstruct the node encoding features corresponding to each node according to the attention weights, so as to obtain the node reconstruction features corresponding to each node. The detection module is used to determine the traffic anomaly detection result for each node based on the difference information between the node reconstruction features and the node encoding features, and according to the attention dispersion degree corresponding to the node determined based on the attention weight.

[0015] Optionally, the generation module, when generating the dynamic heterogeneous graph of the target communication network, is used to: In each of the multiple cycles, traffic data of the target communication network in each cycle is acquired; the traffic data includes: first traffic data of communication between each network entity in the target communication network and other network entities outside the target communication network, and / or second traffic data of communication between each network entity in the target communication network; Based on the traffic data of the target communication network in each period, multiple target network entities are identified, and a dynamic heterogeneous graph based on the communication between the target network entities is generated.

[0016] The generation module, when determining multiple target network entities based on the traffic data of the target communication network in each cycle and generating a dynamic heterogeneous graph based on the communication between the target network entities, is used to: Based on the traffic data for each period, traffic metadata for each period is determined; the traffic metadata includes at least one of the following: traffic identification information, traffic time-series data, traffic statistics data, domain name and Internet Protocol (IP) address pairs, and transport layer security protocol certificates. Based on the traffic metadata of each period, multiple target network entities in each period are identified, and the communication relationships between different target network entities are determined, generating traffic time-series characteristics and traffic statistics characteristics between every two target network entities with communication relationships. Nodes corresponding to the multiple target network entities in each cycle are generated, and edges between different target network entities are generated according to the communication relationship between them. The traffic time sequence characteristics and traffic statistics characteristics between every two target network entities with communication relationship are used to generate the communication traffic characteristics of the corresponding edge.

[0017] Optionally, the encoding model includes: an edge attribute encoder and a node encoder; The encoding module, when using a pre-trained encoding model to encode the dynamic heterogeneous graph and obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph, is used for: The flow characteristics of the edges of each node are encoded using the edge attribute encoder to obtain the flow encoding characteristics of the edges corresponding to each node. The node encoder is used to encode each node to obtain the node encoding features corresponding to each node; wherein, the node encoding features corresponding to each node are obtained by feature aggregation of the stream encoding features of each node, the neighboring nodes of each node, and the edges of each node.

[0018] Optionally, the communication traffic characteristics of each edge corresponding to each node include time-series traffic characteristics and statistical traffic characteristics; The encoding module, when encoding the flow features of the edges of each node using the edge attribute encoder to obtain the flow encoding features of the edges corresponding to each node, is used for: The temporal flow features corresponding to each edge of each node are subjected to a first encoding process to obtain the first stream encoding features corresponding to each edge; as well as, The statistical flow characteristics of each edge corresponding to each node are subjected to a second encoding process to obtain the second flow encoding characteristics corresponding to each edge; The first stream coding feature and the second stream coding feature corresponding to each edge are concatenated to obtain the concatenated coding feature. The concatenated coding feature is then subjected to global nonlinear mapping and dimensional transformation to obtain the stream coding feature corresponding to each edge.

[0019] Optionally, the time-series traffic features include: time-series traffic features corresponding to multiple time windows respectively; The encoding module, when performing a first encoding process on the temporal flow features of each edge corresponding to each node to obtain the first stream encoding features corresponding to each edge, is used for: For each edge corresponding to each node, based on the self-attention mechanism, the temporal flow features corresponding to multiple time windows are encoded to obtain the temporal encoding features corresponding to each edge in multiple time windows. The temporal coding features corresponding to each edge in each time window and the auxiliary statistical features of the corresponding time window are concatenated to obtain the enhanced temporal coding features of each edge in each time window. According to the order of time windows, the enhanced temporal coding features corresponding to each edge in multiple time windows are combined to form a temporal coding feature sequence, and the temporal coding feature sequence is convolved to obtain the first stream coding feature corresponding to each edge.

[0020] Optionally, the encoding module, when performing a second encoding process on the statistical flow features of each edge corresponding to each node to obtain the second flow encoding features corresponding to each edge, is used to: The statistical flow features of each edge corresponding to each node are nonlinearly mapped and dimensionally transformed using a multilayer perceptron to obtain the second flow coding features corresponding to each edge.

[0021] Optionally, the detection module, when determining the traffic anomaly detection result for each node based on the difference information between the node reconstruction features and the node encoding features, and according to the attention dispersion determined based on the attention weight and corresponding to the node, is used to: Based on the differences between the node reconstruction features and the node encoding features, a reconstruction error score is determined; and an attention dispersion score is determined based on the attention weights corresponding to each node and different normal behavior prototypes. Based on the reconstruction error score and the attention distraction score, the normal behavior prototype group to which each node belongs is determined; The reconstruction error score and the attention distraction score are compared with the baseline threshold corresponding to the normal behavior prototype group to which each node belongs; If either the reconstruction error score or the attention dispersion score corresponding to any node is greater than the corresponding baseline threshold, the node is determined to be an abnormal node.

[0022] Optionally, the anomaly detection module is further configured to: for anomaly nodes, based on the difference information corresponding to each edge corresponding to the anomaly node, use a gradient backpropagation mechanism to determine the anomaly contribution degree of each edge corresponding to the anomaly node. The target abnormal edge is determined based on the abnormal contribution of each edge corresponding to the abnormal node.

[0023] Thirdly, an optional implementation of this disclosure also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the first aspect above, or any possible implementation of the first aspect.

[0024] Fourthly, an optional implementation of this disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the first aspect or any possible implementation of the first aspect.

[0025] Fifthly, an optional implementation of this disclosure also provides a computer program product carrying program code, the program code including instructions that can be used to perform the steps of the method as described in the first aspect or any one of the first aspects.

[0026] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0027] This embodiment utilizes a dynamic heterogeneous graph to describe the communication relationships between various network entities in a communication network, enabling more effective modeling of these relationships. Simultaneously, by combining pre-trained neural memory networks with normal behavior prototypes of various normal communication behaviors automatically learned, it ensures that the neural memory network can fully cover the diversity of normal communication patterns. Based on the neural memory network, correlation processing is performed on the encoded features corresponding to each node. The resulting attention weights for each node and different normal behavior prototypes reflect the degree to which the communication behavior of each node is explained by the normal behavior prototypes. Therefore, based on the reconstructed features of the encoded features corresponding to each node and the differences between these features, as well as the attention dispersion represented by the attention weights, the traffic anomaly detection results for each node can be determined more accurately.

[0028] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of a method for detecting abnormal traffic; Figure 2 This is a flowchart illustrating a specific method for encoding dynamic heterogeneous graphs according to an exemplary embodiment of this disclosure; Figure 3 This is a schematic diagram illustrating a specific example of an encoding model according to an exemplary embodiment of this disclosure; Figure 4 This is a schematic diagram illustrating a specific example of the structure of a neural memory network according to an exemplary embodiment of this disclosure; Figure 5 This is a schematic diagram of a computer device illustrated in an exemplary embodiment of the present disclosure; Figure 6This is a schematic diagram of an abnormal traffic detection device illustrated in an exemplary embodiment of this disclosure; Figure 7 This is a specific example of a traffic anomaly detection method illustrated in an exemplary embodiment of this disclosure. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0033] Anomaly detection is a core component of network security and operations, identifying abnormal traffic by recognizing behaviors that deviate from the normal traffic baseline. Conventional anomaly detection methods can be categorized into feature-based, statistical, and machine learning-based approaches. Feature-based methods predefine characteristic rules for known attacks or abnormal traffic, such as specific ports, protocols, packet lengths, and attack signatures, and match real-time traffic against a rule base; a successful match indicates an anomaly. However, with the increasing complexity of network environments and the growing sophistication of attack methods, it is difficult to exhaustively define all characteristic rules for abnormal traffic, making it unsuitable for today's increasingly complex network conditions.

[0034] Statistical detection methods use the statistical characteristics of traffic as a benchmark, identifying anomalies by calculating the normal range of traffic indicators. A statistical baseline is established by first collecting statistical parameters of normal network traffic, such as mean, variance, peak, packet size distribution, and connection threshold. When real-time traffic parameters exceed the baseline's confidence interval (e.g., 95% confidence level), it is considered an anomaly. However, this method requires high accuracy of the baseline, necessitating recalibration when network topology or services change; it is also susceptible to sudden surges in normal traffic, leading to false alarms.

[0035] Machine learning-based detection methods train a model on a large amount of normal and abnormal traffic data, allowing the model to automatically learn normal traffic patterns. The trained model is then used to classify real-time traffic (normal / abnormal). However, in reality, normal network behavior patterns are diverse, and there is a lack of an effective baseline learning mechanism for normal behavior.

[0036] Furthermore, the network entity relationships in the current network environment are complex, including various types of nodes such as IP addresses, domain names, and certificates, making it difficult for traditional methods to effectively model such heterogeneous relationships. Moreover, the detection results lack interpretability, making it difficult for security analysts to quickly pinpoint the root cause of threats.

[0037] Therefore, a method for detecting traffic anomalies in complex network relationships has become an urgent problem to be solved.

[0038] To address the aforementioned issues, this disclosure provides a method for detecting traffic anomalies. The method generates a dynamic heterogeneous graph of a target communication network. This graph includes, for example, nodes corresponding to multiple target network entities, edges describing the communication relationships between each node and its corresponding neighboring nodes, and traffic features corresponding to each edge. Then, using a pre-trained encoding model, the dynamic heterogeneous graph is encoded to obtain encoded features corresponding to each node. Next, based on normal behavior prototypes of various normal communication behaviors learned by a neural memory network, an addressing operation is performed on the encoded features corresponding to each node to obtain attention weights corresponding to each node and different normal behavior prototypes. Based on these attention weights, the encoded features corresponding to each node are reconstructed to obtain reconstructed features for each node. Based on the reconstructed features, the differences between the encoded features, and the attention dispersion represented by the attention weights, the traffic anomaly detection result for each node is determined. In this process, a dynamic heterogeneous graph is used to describe the communication relationships between various network entities, enabling more effective modeling of these relationships. Simultaneously, by combining pre-trained neural memory networks with normal behavior prototypes of various normal communication behaviors automatically learned, the neural memory network ensures it can fully cover the diversity of normal communication patterns. Based on the neural memory network, the encoded features corresponding to each node are processed. The resulting attention weights for each node and different normal behavior prototypes reflect the degree to which the communication behavior of each node is explained by the normal behavior prototypes. Therefore, based on the reconstructed features of the encoded features corresponding to each node, the differences between these features, and the attention dispersion represented by the attention weights, the abnormal traffic detection results for each node can be determined more accurately.

[0039] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0040] To facilitate understanding of this embodiment, a detailed description of the abnormal traffic detection method disclosed in this disclosure is provided first. The execution entity of the abnormal traffic detection method provided in this disclosure is generally a computer device with a certain computing power. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, an in-vehicle device, a wearable device, etc. In some possible implementations, the abnormal traffic detection method can be implemented by a processor calling computer-readable instructions stored in memory.

[0041] The abnormal traffic detection method provided in the embodiments of this disclosure will be described below.

[0042] See Figure 1 The diagram shows a flowchart of an abnormal traffic detection method provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein: S101: Generate a dynamic heterogeneous graph of the target communication network; the dynamic heterogeneous graph includes: nodes corresponding to the plurality of target network entities respectively, edges used to describe the communication relationship between each node and its corresponding neighboring nodes, and traffic features corresponding to each edge; S102: Using a pre-trained encoding model, the dynamic heterogeneous graph is encoded to obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph. S103: Based on the normal behavior prototypes of various normal communication behaviors learned by the neural memory network, an addressing operation is performed on the node encoding features corresponding to each node to obtain the attention weights corresponding to each node and different normal behavior prototypes respectively, and the node encoding features corresponding to each node are reconstructed according to the attention weights to obtain the node reconstruction features corresponding to each node. S104: Based on the difference information between the node reconstruction features and the node encoding features, and according to the attention dispersion degree corresponding to the node determined based on the attention weight, determine the traffic anomaly detection result corresponding to each node.

[0043] The following provides a detailed explanation of S101 to S104.

[0044] Regarding the above S101: In practical implementation, the target communication network is, for example, a specific network within a certain scope determined based on actual needs, such as an organization's internal network, or a network composed of multiple network entities determined based on actual needs. An internal network typically refers to a dedicated network managed autonomously by an organization or institution, accessible only to authorized internal devices; it is also often called an intranet or private network. Its core characteristic is logical isolation from external public networks (such as the Internet), primarily serving internal resource sharing, data transmission, and business collaboration. The internal network's topology, IP address planning, security policies, and device configurations are all independently controlled by the organization and are not directly constrained by external network operators or public network rules.

[0045] Taking an internal network as an example, a communication network may include at least some of the following network entities: a1: Terminal entities, such as personal terminals like PCs, laptops, mobile phones, and tablets, and dedicated terminals like industrial control terminals, cash registers, access control and card-swiping devices, and surveillance cameras.

[0046] a2: Forwarding and interconnection entities, such as core layer devices like core switches and routers, aggregation switches, access layer devices like access switches and wireless access points.

[0047] a3: Service and storage entities, such as servers, network-attached storage, storage area networks, and other storage devices.

[0048] a4: Security and control entities, such as firewalls, Network Address Translation (NAT) devices, Virtual Private Network (VPN) gateways, etc.

[0049] When generating a dynamic heterogeneous graph of the target communication network, communication relationships may exist between network entities within the target communication network, or between network entities within the target communication network and other communication entities outside the target communication network. Therefore, each target network entity in the dynamic structure graph may include, for example, network entities within the target communication network and other network entities outside the target communication network. These other network entities communicate with network entities within the target communication network, thus serving as nodes in the dynamic structure.

[0050] The dynamic heterogeneous graph described in this embodiment includes: nodes corresponding to the plurality of target network entities, edges used to describe the communication relationship between each node and its corresponding neighboring nodes, and traffic characteristics corresponding to each edge.

[0051] For example, this disclosure provides a specific example of nodes, edges, and corresponding traffic characteristics in a dynamic heterogeneous graph, wherein: (1) Based on the communication semantics of the attack, this disclosure defines the following four types of nodes: Internal IP Nodes Based on the RFC 1918 address space (Address Allocation for Private Internets, a private IPv4 address space standard defined by the IETF), it automatically identifies internal network hosts. Each node is uniquely identified by an Internet Protocol (IP) address and associated with metadata such as device type and network location.

[0052] External IP Node : Identifies the public IP address used for communication with the internal network, and records attributes such as geographical location and reputation score.

[0053] Domain Node It extracts canonical domain names from Domain Name System (DNS) query logs and associates them with WHOIS information (referring to public records stored in the domain name registration database), domain age, and other characteristics.

[0054] Certificate Node It is constructed based on the SHA256 hash value of the TLS handshake certificate, and records certificate attributes such as issuer, validity period, and key algorithm.

[0055] (2) In order to describe the complex interactions between the above-mentioned network entities, the embodiments of this disclosure define the following five types of core edge relationships: Outbound access relationship This indicates a TLS connection initiated by the internal host.

[0056] Inbound access relationship This indicates an externally initiated Transmission Control Protocol (TCP) connection attempt.

[0057] Internal access relationships This indicates communication between hosts on the internal network.

[0058] DNS resolution relationship Record DNS query behavior.

[0059] Certificate usage relationship Establish the association between the service endpoint and the identity credentials.

[0060] (3) Construct traffic features containing statistical and temporal characteristics for each outbound access edge, inbound access edge, and internal access edge: Traffic statistics features include at least one of the following: total number of bytes in the stream, number of data packets, stream duration, and ratio of uplink to downlink traffic.

[0061] Traffic timing characteristics include, for example, packet arrival time sequence, packet size distribution, and changes in packet flags.

[0062] Specifically, embodiments of this disclosure also provide a method for generating a dynamic heterogeneous graph of a target communication network, including: In each of the multiple cycles, traffic data of the target communication network in each cycle is acquired; the traffic data includes: first traffic data of communication between each network entity in the target communication network and other network entities outside the target communication network, and / or second traffic data of communication between each network entity in the target communication network; Based on the traffic data of the target communication network in each period, multiple target network entities are identified, and a dynamic heterogeneous graph based on the communication between the target network entities is generated.

[0063] This ensures that the dynamic heterogeneous graph is generated periodically, or regenerated when the communication relationships between different target network entities represented by the original dynamic heterogeneous graph change, or updated based on the original dynamic heterogeneous graph. This enables a rapid response to abnormal communication and improves network security.

[0064] Furthermore, embodiments of this disclosure also provide a specific method for determining multiple target network entities based on traffic data of the target communication network in each period, and generating a dynamic heterogeneous graph based on the communication between the target network entities, including: Based on the traffic data for each period, traffic metadata for each period is determined; the traffic metadata includes at least one of the following: traffic identification information, traffic time-series data, traffic statistics data, domain name and Internet Protocol (IP) address pairs, and transport layer security protocol certificates; Based on the traffic metadata of each period, multiple target network entities in each period are identified, and the communication relationships between different target network entities are determined, generating traffic time-series characteristics and traffic statistics characteristics between every two target network entities with communication relationships. Nodes corresponding to the multiple target network entities in each cycle are generated, and edges between different target network entities are generated according to the communication relationship between them. The traffic time sequence characteristics and traffic statistics characteristics between every two target network entities with communication relationship are used to generate the communication traffic characteristics of the corresponding edge.

[0065] Here, traffic data may include, for example, traffic mirrors generated from communication between network entities, and DNS logs. When generating traffic metadata based on traffic data, operations such as protocol restoration and TLS feature extraction can be performed based on the traffic mirrors, and DNS logs can be parsed to obtain traffic metadata.

[0066] Traffic metadata includes, for example, traffic identification information such as source IP address, destination IP address, source port number, destination port number, and transport layer protocol. Then, based on the traffic metadata, multiple target network entities corresponding to the identified and defined node types are identified. Based on the communication data between different target network entities, the communication relationships between them are determined, and traffic timing characteristics and traffic statistics characteristics are generated for each pair of target network entities with communication relationships.

[0067] Next, nodes corresponding to the target network entities are generated, and corresponding descriptive information is added to the nodes corresponding to each target network entity. Based on the communication relationships between different target network entities, edges corresponding to the various types of edges defined between the target network entities are generated, and corresponding traffic time-series features and traffic statistics features are added to each edge as the corresponding traffic features of each edge, thus forming a dynamic heterogeneous graph.

[0068] Regarding S102 above: In practice, the pre-trained encoding model is obtained, for example, by training data from a large-scale network. Encoding models include, for example, edge attribute encoders and node encoders.

[0069] See Figure 2 As shown, when using an encoding model to encode the dynamic heterogeneous graph and obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph, the following method can be used, for example: S201: The flow characteristics of the edges of each node are encoded using the edge attribute encoder to obtain the flow coding characteristics of the edges corresponding to each node.

[0070] In specific implementations, when using an edge attribute encoder to encode the flow characteristics of the edges of each node, the process may include steps 1 to 3 as follows: Step 1: Perform a first encoding process on the temporal flow features of each edge corresponding to each node to obtain the first stream encoding features corresponding to each edge.

[0071] Here, when performing the first encoding process on the temporal flow characteristics of each edge corresponding to each node, the following method can be used, for example: For each edge corresponding to each node, based on the self-attention mechanism, the temporal flow features corresponding to multiple time windows are encoded to obtain the temporal encoding features corresponding to each edge in multiple time windows. The temporal coding features corresponding to each edge in each time window and the auxiliary statistical features of the corresponding time window are concatenated to obtain the enhanced temporal coding features of each edge in each time window. According to the order of the time windows, the enhanced temporal coding features corresponding to each edge in multiple time windows are sorted and combined to form a temporal coding feature sequence. The temporal coding feature sequence is then convolved to obtain the first stream coding feature corresponding to each edge.

[0072] For example, an edge attribute encoder may include: ① a stream temporal feature encoder, ② a feature fusion module, and ③ a window temporal feature encoder.

[0073] Among them, ① the flow temporal feature encoder is specifically used to encode the temporal flow features corresponding to multiple time windows for each edge corresponding to each node based on the self-attention mechanism, so as to obtain the temporal coding features corresponding to each edge in multiple time windows.

[0074] Specifically, stream temporal feature encoder For example, based on the Transformer architecture, a self-attention mechanism is used to capture long-range temporal dependencies, specifically handling fine-grained temporal features within a single connection. This stream temporal feature encoder... For any time window Time-series flow characteristics Encoding process is performed to obtain the time window. Corresponding temporal coding features For example, it satisfies the following formula (1): (1) in, Indicates a time window. Temporal characteristics within a time window; This represents the temporal coding characteristics of a single connection (each stream). Indicates the number of time windows.

[0075] Among them, the stream temporal feature encoder The encoder is pre-trained using large-scale network connection time-series data, and the pre-training process uses large-scale network connection data to improve generalization ability.

[0076] ② Feature fusion module, specifically used to concatenate the temporal coding features corresponding to each edge in each time window and the auxiliary statistical features of the corresponding time window to obtain the enhanced temporal coding features of each edge in each time window.

[0077] Specifically, the temporal coding features of each stream in each time window Auxiliary statistical feature vectors within the corresponding time window The features are then concatenated to form enhanced temporal coding features for each time window. Satisfy the following formula (2): (2) in, This indicates splicing / merging.

[0078] Here, the auxiliary statistical feature vector within the time window For example, it can be obtained by statistically analyzing the communication data of the corresponding edge within the time window, which may include at least one of the following: the total number of bytes in the flow corresponding to the edge within the time window, the number of data packets, the flow duration, and the ratio of uplink to downlink traffic.

[0079] ③ A window-based temporal feature encoder, for example, sorts and combines the enhanced temporal coding features corresponding to each edge in multiple time windows according to the order of time windows to form a temporal coding feature sequence, and performs convolution processing on the temporal coding feature sequence to obtain the first stream coding feature corresponding to each edge.

[0080] For example, the number of time windows is ,Will The enhanced temporal coding features corresponding to each time window are sorted and combined to obtain the temporal coding feature sequence as follows: .

[0081] This temporal encoded feature sequence contains both enhanced features of a single stream and preserves the temporal correlations between multiple streams. Subsequently, a temporal convolutional network encoder is employed. Encode the temporal coding feature sequence. The temporal convolutional network encoder expands the receptive field through causal convolution and dilated convolution, effectively capturing local temporal patterns while maintaining parallel computing efficiency. The encoding process satisfies the following formula (3): (3) Step 2: Perform a second encoding process on the statistical flow features of each edge corresponding to each node to obtain the second flow encoding features corresponding to each edge.

[0082] In practical implementation, the statistical flow characteristics of each edge corresponding to each node are as follows: For example, it can be represented as: .

[0083] in, This represents a statistical feature dimension, covering at least one of the following edge attributes: total number of bytes in the stream, number of data packets, stream duration, and ratio of uplink to downlink traffic. Indicates the first The values ​​of each traffic statistical feature.

[0084] When performing a second encoding process on the statistical flow characteristics of each edge corresponding to each node, a statistical feature encoder can be used, for example. This statistical feature encoder may include, for example, a multilayer perceptron. Using a multilayer perceptron The statistical flow characteristics of each edge corresponding to each node are subjected to nonlinear mapping and dimensional transformation to obtain the second flow coding features corresponding to each edge.

[0085] Specifically, the obtained second-stream coding features For example, it can be expressed as the following formula (4): (4) Step 3: Concatenate the first stream coding feature and the second stream coding feature corresponding to each edge to obtain the concatenated coding feature, and perform global nonlinear mapping and dimensional transformation on the concatenated coding feature to obtain the stream coding feature corresponding to each edge.

[0086] In practical implementation, a global feature fusion unit can be used, for example, to perform the aforementioned fusion process on the first-stream encoded features and second-stream encoded features corresponding to each edge. The global feature fusion unit includes a concatenation unit for concatenating the first-stream encoded features and second-stream encoded features to obtain concatenated encoded features. The global feature fusion unit may also include, for example, a global multilayer perceptron. The splicing and encoding features are deeply fused and their dimensions are adjusted.

[0087] The specific encoding process can be represented by the following formula (5): (5) in, Represents a node and nodes Stream coding features corresponding to the edges between .

[0088] Represents a node and nodes The first-stream encoded features of the edges between them. Represents a node and nodes The second-stream encoded features of the edges between them. This indicates splicing / merging.

[0089] S202: Using the node encoder, each node is encoded to obtain the node encoding features corresponding to each node; wherein, the node encoding features corresponding to each node are obtained by feature aggregation of the stream encoding features of each node, the neighboring nodes of each node, and the edges of each node.

[0090] In specific implementations, the node encoder may include, for example, an extended relational graph convolutional network. Specifically, the node encoder has multiple network layers; for each node... In its first Embedding features of each network layer Satisfy the following formulas (6) and (7): (6) (7) Represents a node In the Layer embedding features; Represents a node In the Layer embedding features; Represents a node In the Layer embedding features; Representation and Node Nodes with any type of edge between them The relationship in the Embedding features of layers.

[0091] Representing relations A dedicated, learnable attention vector, also known as a network intrinsic parameter in a relational graph convolutional network. Here, "relationship" refers to... For example, it includes the five types of edge relationships defined above, such as outbound access relationships, inbound access relationships, internal access relationships, DNS resolution relationships, and certificate usage relationships.

[0092] It is a node and its neighboring nodes In relationship bottom edge Attention weights.

[0093] It represents the weight matrix of various types of edge relationships, and learns the propagation patterns of different semantic relationships (communication relationships). Here, semantic relationships refer to the descriptive information of communication behavior in network communication scenarios and related information described by natural language.

[0094] This indicates a vector concatenation operation that fuses node features with edge features.

[0095] The normalization constant representing the node degree, for the relation The normalization constant is usually taken as .

[0096] Represents nodes Corresponding representation relationship The number of edges.

[0097] Next, the node output from the last network layer of the node encoder will be... The embedding features of the node The corresponding node encoding features .

[0098] like Figure 3As shown, this disclosure provides a specific example of an encoding model, including: an edge attribute encoder and a node attribute encoder.

[0099] The edge attribute encoder includes: a time-series feature encoder, a feature fusion module, and a window time-series feature encoder connected in sequence.

[0100] The sequentially connected temporal feature encoder, feature fusion module, and window temporal feature encoder are used to perform a first encoding process on the temporal flow features of each edge corresponding to each node, so as to obtain the first stream encoding features corresponding to each edge.

[0101] The edge attribute encoder further includes a statistical feature encoder, which performs a second encoding process on the statistical flow features of each edge corresponding to each node to obtain the second flow encoding features corresponding to each edge.

[0102] The edge attribute encoder also includes a global feature fusion unit connected to the window temporal feature encoder and the statistical feature encoder, which is used to concatenate the first stream coding feature and the second stream coding feature corresponding to each edge to obtain the concatenated coding feature, and to perform global nonlinear mapping and dimensional transformation on the concatenated coding feature to obtain the stream coding feature corresponding to each edge.

[0103] The node attribute encoder is used to encode each node to obtain the node encoding features corresponding to each node.

[0104] Regarding the above S103: A neural memory network is a trainable parameter matrix. ,in This indicates the number of memory slots (MemoryBank) (i.e., the normal behavior prototype). The dimension is the same as the node's node encoding feature. Each row vector This represents a learned prototype of normal behavior (such as "Web server", "database server", etc.).

[0105] Figure 4 A specific example of the structure of a neural memory network is shown. Figure 4 middle, Represents a node The corresponding node encoding features. Indicates the node Corresponding node encoding features and various normal behavioral prototypes Perform an addressing operation to obtain the node. Corresponding node encoding features and various normal behavioral prototypes Attention weights between Then, based on the node encoding features... and various normal behavioral prototypes Attention weights between The node encoding features corresponding to each node are reconstructed to obtain the node reconstruction features corresponding to each node.

[0106] Specifically, when reconstructing the node encoding features corresponding to a node to obtain the node reconstruction features corresponding to each node, for example, the node encoding features can be used... and various normal behavioral prototypes Attention weights between , encoding features for nodes Perform a weighted summation to obtain the nodes. Corresponding reconstructed node reconstruction features .

[0107] Specifically, the nodes Corresponding node encoding features and various normal behavioral prototypes Perform an addressing operation to obtain the node. Corresponding node encoding features and various normal behavioral prototypes Attention weights between Satisfy the following formula (8): (8) This represents cosine similarity, which is also the addressing operation.

[0108] This is a temperature hyperparameter used to control the smoothness of the attention distribution. Weights Reflects the nodes The behavior was prototype The degree of explanation; the higher the value, the more similar the similarity.

[0109] The read operation, based on attention weights, reconstructs the feature data of nodes to obtain the reconstructed node features. Among these features, the node... Corresponding reconstructed node reconstruction features The following formula (9) must be satisfied: (9) in, Nodes based on normal behavior prototypes The best normal explanation for the behavior. Because the neural memory module is shaped only by the dominant normal nodes during training, abnormal nodes, unable to align with any prototype, have scattered attention weights, leading to reconstructed embeddings. With the original embedding This results in significant reconstruction errors.

[0110] In another embodiment of this disclosure, the loss function used when training the neural memory network includes reconstruction loss and separation loss to ensure that the memory prototypes fully cover the diversity of normal patterns and maintain the distinguishability between prototypes. The designed loss functions are as shown in formulas (10) and (11): (10) (11) Indicates the losses incurred during reconstruction; Indicates the number of nodes; Indicates separation loss; This indicates the number of normal behavior prototypes.

[0111] The loss function is robust to outliers: the reconstruction loss of outlier nodes is large, but due to the dispersion of attention, the gradient effect on any specific prototype is weak, so the memory prototype is not easily contaminated by outliers.

[0112] Regarding S104 above: After obtaining the attention weights corresponding to each node and different normal behavior prototypes, as well as the node reconstruction features corresponding to each node, based on the above steps, the traffic anomaly detection result corresponding to each node is determined in the following way: Based on the differences between the node reconstruction features and the node encoding features, a reconstruction error score is determined. ; In addition, an attention distraction score is determined based on the attention weights corresponding to each node and different normal behavior prototypes.

[0113] Based on the reconstruction error score and the attention distraction score, the normal behavior prototype group to which each node belongs is determined; The reconstruction error score and the attention distraction score are compared with the baseline threshold corresponding to the normal behavior prototype group to which each node belongs; If either the reconstruction error score or the attention dispersion score corresponding to any node is greater than the corresponding baseline threshold, the node is determined to be an abnormal node.

[0114] In practical implementation, the reconstruction error fraction Satisfy the following formula (11): (11) Distraction score Satisfy the following formula (12): (12) When determining the target normal behavior prototype to which each node belongs based on the attention weights corresponding to each node and different normal behavior prototypes, the maximum value of the attention weights corresponding to each node can be determined, and the normal behavior prototype corresponding to the maximum value of the attention weight can be determined as the target normal behavior prototype to which the node belongs.

[0115] For example,

[0116] For each prototype group Maintain the reconstruction error score and attention dispersion score of its member nodes. For the heavy-tailed distribution characteristics of the reconstruction error score and attention dispersion score, a dynamic threshold can be set, for example, using the Peaks OverThreshold (POT) method.

[0117] Among them, the dynamic threshold determined by the POT method Satisfy the following formula (13): (13) This represents the initial threshold, which can be automatically determined through the average excess graph to ensure accurate identification of extreme anomalies. The final calculated anomaly threshold is defined as the data exceeding [a certain threshold]. The value will be considered abnormal. This is the scale parameter of the generalized Pareto distribution, which controls the degree of diffusion of the distribution. The shape parameter of the generalized Pareto distribution determines the thickness of the tail. This represents the total number of samples in the sliding window. To exceed the initial threshold The number of samples, i.e. the number of tail samples. For the confidence level set by the user, it means that it is desired to set the threshold at a position where, theoretically, only The extreme values ​​of the proportion will exceed this position.

[0118] For each prototype group Reconstruction error fraction aggregation within a certain time window and attention distraction score set Do not calculate the corresponding dynamic threshold , respectively, serve as their baselines.

[0119] When determining node anomalies, both its group affiliation and multi-dimensional anomaly scores are considered. For the node to be detected... First, determine its group. Get the current dynamic threshold of this group. .

[0120] like or , representing a node abnormal.

[0121] After detecting node anomalies, a gradient backpropagation mechanism is used to analyze the causal contribution of each edge to the node anomaly and filter potential anomalous edges. For anomalous nodes... Define edge The causal contribution satisfies the following formula (14): (14) in, Represents a node In the Layer embedding features; The activation function term ensures that only the contribution of nodes with higher anomaly levels is considered; It is the edge Attention weights in the message passing process. It is an edge node The attention weights of each edge during message passing.

[0122] Based on the calculated causal contribution, the system automatically identifies the top N edges with the largest contribution as potential outliers.

[0123] The edge corresponding to this anomaly is also the flow that may pose a security risk.

[0124] See Figure 7 As shown in the embodiments of this disclosure, a specific example of a method for detecting traffic anomalies is also provided, including: Step 1: Data Acquisition and Processing: Collect data, including traffic mirroring and DNS logs. Process the collected data, including protocol reconstruction, TLS feature extraction, and DNS log parsing, to obtain traffic metadata, including: the traffic's five-tuple (i.e., traffic identification information), time-series characteristics, statistical characteristics, domain-IP pairs, and TLS certificates.

[0125] Step 2: Use the traffic metadata obtained in Step 1 to construct a heterogeneous graph, including entity identification, relationship identification and edge attribute construction, and then generate a dynamic heterogeneous graph.

[0126] Step 3: Process the dynamic heterogeneous graph using the Dynamic Heterogeneous Graph Memory Network (DH-GMN) model. The DH-GMN model includes an edge attribute encoder, a node encoder, and a neural memory network. The prototype reconstruction error score and the prototype dispersion score (attention dispersion score) are obtained.

[0127] Step 4: Based on the prototype reconstruction error score (reconstruction error score) and prototype dispersion score (attention dispersion score), perform multi-dimensional dynamic limit management and anomaly detection. This includes: multi-dimensional scoring, prototype grouping dynamic threshold calculation, prototype grouping, anomaly determination, and outputting detection results including anomaly alarms and interpretability analysis. The interpretability analysis results include nodes with anomalies and edges that contribute significantly to the anomalies.

[0128] Corresponding to the embodiments of the aforementioned abnormal flow detection method, this disclosure also provides embodiments of an abnormal flow detection device.

[0129] Embodiments of the traffic anomaly detection device disclosed herein can be applied to computer devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the computer device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 5 The diagram shown is a hardware structure diagram of a computer device housing the traffic anomaly detection device of this disclosure, except... Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, the computer device in which the device is located in the embodiment may also include other hardware depending on the actual function of the traffic anomaly detection device, which will not be described in detail here.

[0130] Please refer to Figure 6 The abnormal traffic detection device provided in this embodiment includes: The generation module 61 is used to generate a dynamic heterogeneous graph of the target communication network; the dynamic heterogeneous graph includes: nodes corresponding to the plurality of target network entities respectively, edges used to describe the communication relationship between each node and its corresponding neighboring nodes, and traffic features corresponding to each edge; Encoding module 62 is used to encode the dynamic heterogeneous graph using a pre-trained encoding model to obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph. The reconstruction module 53 is used to perform addressing operations on the node encoding features corresponding to each node based on the normal behavior prototypes of various normal communication behaviors learned by the neural memory network, to obtain the attention weights corresponding to each node and different normal behavior prototypes respectively, and to perform reconstruction processing on the node encoding features corresponding to each node according to the attention weights, to obtain the node reconstruction features corresponding to each node. The detection module 64 is used to determine the traffic anomaly detection result for each node based on the difference information between the node reconstruction features and the node encoding features, and according to the attention dispersion degree corresponding to the node determined based on the attention weight.

[0131] Optionally, the generation module, when generating the dynamic heterogeneous graph of the target communication network, is used to: In each of the multiple cycles, traffic data of the target communication network in each cycle is acquired; the traffic data includes: first traffic data of communication between each network entity in the target communication network and other network entities outside the target communication network, and / or second traffic data of communication between each network entity in the target communication network; Based on the traffic data of the target communication network in each period, multiple target network entities are identified, and a dynamic heterogeneous graph based on the communication between the target network entities is generated.

[0132] The generation module, when determining multiple target network entities based on the traffic data of the target communication network in each cycle and generating a dynamic heterogeneous graph based on the communication between the target network entities, is used to: Based on the traffic data for each period, traffic metadata for each period is determined; the traffic metadata includes at least one of the following: traffic identification information, traffic time-series data, traffic statistics data, domain name and Internet Protocol (IP) address pairs, and transport layer security protocol certificates. Based on the traffic metadata of each period, multiple target network entities in each period are identified, and the communication relationships between different target network entities are determined, generating traffic time-series characteristics and traffic statistics characteristics between every two target network entities with communication relationships. Nodes corresponding to the multiple target network entities in each cycle are generated, and edges between different target network entities are generated according to the communication relationship between them. The traffic time sequence characteristics and traffic statistics characteristics between every two target network entities with communication relationship are used to generate the communication traffic characteristics of the corresponding edge.

[0133] Optionally, the encoding model includes: an edge attribute encoder and a node encoder; The encoding module, when using a pre-trained encoding model to encode the dynamic heterogeneous graph and obtain the node encoding features corresponding to each node in the dynamic heterogeneous graph, is used for: The flow characteristics of the edges of each node are encoded using the edge attribute encoder to obtain the flow encoding characteristics of the edges corresponding to each node. The node encoder is used to encode each node to obtain the node encoding features corresponding to each node; wherein, the node encoding features corresponding to each node are obtained by feature aggregation of the stream encoding features of each node, the neighboring nodes of each node, and the edges of each node.

[0134] Optionally, the communication traffic characteristics of each edge corresponding to each node include time-series traffic characteristics and statistical traffic characteristics; The encoding module, when encoding the flow features of the edges of each node using the edge attribute encoder to obtain the flow encoding features of the edges corresponding to each node, is used for: The temporal flow features corresponding to each edge of each node are subjected to a first encoding process to obtain the first stream encoding features corresponding to each edge; as well as, The statistical flow characteristics of each edge corresponding to each node are subjected to a second encoding process to obtain the second flow encoding characteristics corresponding to each edge; The first stream coding feature and the second stream coding feature corresponding to each edge are concatenated to obtain the concatenated coding feature. The concatenated coding feature is then subjected to global nonlinear mapping and dimensional transformation to obtain the stream coding feature corresponding to each edge.

[0135] Optionally, the time-series traffic features include: time-series traffic features corresponding to multiple time windows respectively; The encoding module, when performing a first encoding process on the temporal flow features of each edge corresponding to each node to obtain the first stream encoding features corresponding to each edge, is used for: For each edge corresponding to each node, based on the self-attention mechanism, the temporal flow features corresponding to multiple time windows are encoded to obtain the temporal encoding features corresponding to each edge in multiple time windows. The temporal coding features corresponding to each edge in each time window and the auxiliary statistical features of the corresponding time window are concatenated to obtain the enhanced temporal coding features of each edge in each time window. According to the order of time windows, the enhanced temporal coding features corresponding to each edge in multiple time windows are combined to form a temporal coding feature sequence, and the temporal coding feature sequence is convolved to obtain the first stream coding feature corresponding to each edge.

[0136] Optionally, the encoding module, when performing a second encoding process on the statistical flow features of each edge corresponding to each node to obtain the second flow encoding features corresponding to each edge, is used to: The statistical flow features of each edge corresponding to each node are nonlinearly mapped and dimensionally transformed using a multilayer perceptron to obtain the second flow coding features corresponding to each edge.

[0137] Optionally, the detection module, when determining the traffic anomaly detection result for each node based on the difference information between the node reconstruction features and the node encoding features, and according to the attention dispersion determined based on the attention weight and corresponding to the node, is used to: Based on the differences between the node reconstruction features and the node encoding features, a reconstruction error score is determined; and an attention dispersion score is determined based on the attention weights corresponding to each node and different normal behavior prototypes. Based on the reconstruction error score and the attention distraction score, the normal behavior prototype group to which each node belongs is determined; The reconstruction error score and the attention distraction score are compared with the baseline threshold corresponding to the normal behavior prototype group to which each node belongs; If either the reconstruction error score or the attention dispersion score corresponding to any node is greater than the corresponding baseline threshold, the node is determined to be an abnormal node.

[0138] Optionally, the anomaly detection module is further configured to: for anomaly nodes, based on the difference information corresponding to each edge corresponding to the anomaly node, use a gradient backpropagation mechanism to determine the anomaly contribution degree of each edge corresponding to the anomaly node. The target abnormal edge is determined based on the abnormal contribution of each edge corresponding to the abnormal node.

[0139] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0140] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0141] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the traffic anomaly detection method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0142] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the traffic anomaly detection method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0143] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0144] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0145] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0146] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0147] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0148] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0149] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0150] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0151] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0152] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method of anomaly detection of traffic, characterized by, The method comprises: generating a dynamic heterogeneous graph of a target communication network; the dynamic heterogeneous graph comprises nodes corresponding to a plurality of target network entities respectively, edges for describing a communication relationship between each node and a corresponding neighbor node, and a traffic feature corresponding to each edge; using a pre-trained encoding model to perform encoding processing on the dynamic heterogeneous graph to obtain node encoding features corresponding to each node in the dynamic heterogeneous graph respectively; based on normal behavior prototypes of a plurality of normal communication behaviors learned by a neural memory network, performing addressing operation on the node encoding features corresponding to each node to obtain attention weights corresponding to each node and different normal behavior prototypes respectively, and performing reconstruction processing on the node encoding features corresponding to each node according to the attention weights to obtain node reconstruction features corresponding to each node; based on the node reconstruction features and difference information between the node encoding features, and according to an attention dispersion degree corresponding to the node determined based on the attention weights, determining a traffic anomaly detection result corresponding to each node.

2. The method of claim 1, wherein, The method comprises: in each period of a plurality of periods, obtaining traffic data of a target communication network in each period; the traffic data comprises first traffic data of each network entity in the target communication network communicating with other network entities outside the target communication network, and / or second traffic data of each network entity in the target communication network communicating with each other; determining a plurality of target network entities according to the traffic data of the target communication network in each period, and generating a dynamic heterogeneous graph based on mutual communication between the target network entities.

3. The method of claim 2, wherein, The method comprises: based on the traffic data of each period, determining traffic metadata of each period; the traffic metadata comprises at least one of identification information of traffic, traffic timing data, traffic statistical data, a domain name and an Internet Protocol (IP) address pair, and a Transport Layer Security (TLS) certificate; based on the traffic metadata of each period, identifying a plurality of target network entities of each period, determining a communication relationship between different target network entities, and generating traffic timing features and traffic statistical features between each two target network entities with the communication relationship; generating nodes corresponding to the plurality of target network entities of each period respectively, generating edges between different target network entities according to the communication relationship between different target network entities, and generating communication traffic features of the corresponding edges based on the traffic timing features and the traffic statistical features between each two target network entities with the communication relationship.

4. The method according to any one of claims 1 to 3, characterized in that, The encoding model comprises an edge attribute encoder and a node encoder. The method comprises: The edge attribute encoder is used to encode the traffic features of the edges of each node to obtain the flow encoding features of the edges corresponding to each node. The node encoder is used to encode each node to obtain the node encoding features corresponding to each node, wherein the node encoding features corresponding to each node are obtained by aggregating the features of each node, the neighbor nodes of each node, and the flow encoding features of the edges of each node.

5. The method of claim 4, wherein, The communication traffic features of each edge corresponding to each node include time-series traffic features and statistical traffic features. The edge attribute encoder is used to encode the traffic features of the edges of each node to obtain the flow encoding features of the edges corresponding to each node. The time-series traffic features of each edge corresponding to each node are first encoded to obtain the first flow encoding features corresponding to each edge. The statistical traffic features of each edge corresponding to each node are second encoded to obtain the second flow encoding features corresponding to each edge. The first flow encoding features and the second flow encoding features corresponding to each edge are spliced to obtain spliced encoding features, and the spliced encoding features are globally nonlinearly mapped and dimensionally transformed to obtain the flow encoding features corresponding to each edge. The time-series traffic features include time-series traffic features corresponding to multiple time windows.

6. The method of claim 5, wherein, The time-series traffic features of each edge corresponding to each node are first encoded to obtain the first flow encoding features corresponding to each edge, including: For each edge corresponding to each node, the time-series traffic features corresponding to multiple time windows are encoded based on a self-attention mechanism to obtain time-series encoding features of the edge corresponding to multiple time windows. The time-series encoding features of each edge corresponding to each time window and the auxiliary statistical features of the corresponding time window are spliced to obtain enhanced time-series encoding features of each edge in each time window. The enhanced time-series encoding features of each edge corresponding to multiple time windows are combined in the order of the time windows to form a time-series encoding feature sequence, and the time-series encoding feature sequence is convoluted to obtain the first flow encoding features corresponding to each edge. The statistical traffic features of each edge corresponding to each node are second encoded to obtain the second flow encoding features corresponding to each edge, including:

7. The method according to claim 5 or 6, characterized in that, The statistical traffic features of each edge corresponding to each node are nonlinearly mapped and dimensionally transformed by a multi-layer perception to obtain the second flow encoding features corresponding to each edge. The difference information between the node reconstruction features and the node encoding features is used to determine the traffic anomaly detection results corresponding to each node based on the attention dispersion degrees corresponding to the nodes determined based on the attention weights, including:

8. The method of claim 1, wherein, ​ determine a reconstruction error score based on the difference information between the node reconstruction feature and the node encoding feature, and determine an attention dispersion score based on the attention weight corresponding to each node and different normal behavior prototypes respectively; determine a normal behavior prototype group to which each node belongs based on the reconstruction error score and the attention dispersion score; compare the reconstruction error score and the attention dispersion score with a baseline threshold corresponding to the normal behavior prototype group to which each node belongs respectively; determine a node as an abnormal node if any of the reconstruction error score and the attention dispersion score corresponding to the node is greater than the corresponding baseline threshold.

9. The method of claim 8, wherein, The method further comprises: for an abnormal node, determining abnormal contribution degrees of edges corresponding to the abnormal node based on difference information corresponding to each edge of the abnormal node by using a gradient back propagation mechanism; determine a target abnormal edge based on the abnormal contribution degrees of the edges corresponding to the abnormal node.

10. A flow rate anomaly detection device, characterized in that, The method comprises: a generation module configured to generate a dynamic heterogeneous graph of a target communication network; the dynamic heterogeneous graph comprises nodes corresponding to a plurality of target network entities respectively, edges for describing communication relationships between each node and a corresponding neighbor node, and traffic features corresponding to each edge; an encoding module configured to encode the dynamic heterogeneous graph by using a pre-trained encoding model to obtain node encoding features corresponding to each node in the dynamic heterogeneous graph respectively; a reconstruction module configured to perform addressing operation on the node encoding features corresponding to each node based on normal behavior prototypes of a plurality of normal communication behaviors learned by a neural memory network, to obtain attention weights corresponding to each node and different normal behavior prototypes respectively, and to perform reconstruction processing on the node encoding features corresponding to each node based on the attention weights to obtain node reconstruction features corresponding to each node respectively; a detection module configured to determine a traffic anomaly detection result corresponding to each node based on difference information between the node reconstruction features and the node encoding features, and based on an attention dispersion corresponding to the node determined based on the attention weights.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the traffic anomaly detection method of any one of claims 1-9.

12. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the traffic anomaly detection method of any one of claims 1-9.

Images