Unknown threat detection method and system

Abstract

The application provides an unknown threat detection method and system. The method obtains network telemetry data of a network to be detected, divides the network telemetry data into multiple time slices according to a preset multi-time scale, constructs a connection graph based on the network telemetry data for each time slice, extracts at least one graph feature based on the connection graph, determines an anomaly score according to the graph feature, and generates an alarm evidence package and outputs alarm information when the anomaly score meets a preset alarm condition, so that the alarm has reviewability and traceability, reduces false positives caused by legal scanning roles, and improves alarm handling efficiency.

Description

Technical Field

[0001] This application relates to security detection technology, and more particularly to a method and system for detecting unknown threats. Background Technology

[0002] Existing unknown threat detection methods based on network telemetry data often use statistical thresholds for single time windows, blacklist features, or single-point host behavior analysis to identify service probing behaviors before intranet scanning and lateral movement. They typically use indicators such as the number of connections, the number of failures, the number of destination addresses, or the number of destination ports per unit time as the basis for judgment.

[0003] Then, there are generally high numbers of legitimate access behaviors in enterprise intranets, such as asset inventory, vulnerability assessment, monitoring and management, operation and maintenance jumpers and centralized management. At the same time, the proportion of encryption and east-west traffic in the network is increasing, and business access relationships are changing dynamically. This makes it easy for detection methods that rely solely on short-term statistical thresholds or fixed rules to have high false alarms, high false negatives, incomplete alarm evidence and difficulty in supporting handling.

[0004] The specific reason for the aforementioned problems lies in the fact that traditional detection methods are mostly based on the local statistical characteristics of a single-source node within a short time slice. Attackers can dilute abnormal signals through low-frequency probing and multi-source distribution, making it difficult for indicators such as the number of connections and the failure rate to exceed thresholds within any short time slice. Furthermore, service probing at the Layer 4 network side mainly manifests as connection results such as successful handshake, reset failure, timeout failure, and rejection failure. Normal services also experience failed connections due to configuration errors, service jitter, health checks, and policy rejections, thus making failure characteristics not directly equivalent to malice. Summary of the Invention

[0005] This application provides a method and system for detecting unknown threats, so as to make alarms verifiable and traceable, reduce false alarms and improve alarm handling efficiency.

[0006] Firstly, this application provides a method for detecting unknown threats, including: Acquire network telemetry data of the network to be tested; The network telemetry data is divided into multiple time slices according to a preset multi-time scale; For each of the aforementioned time slices, a connectivity graph is constructed based on the network telemetry data. The connectivity graph includes multiple nodes and edges connecting the nodes. The nodes are used to represent network entities, and the edges are used to represent network connection events. Based on the connectivity graph, at least one graph feature is extracted; Anomaly scores are determined based on the graph features; When the anomaly score meets the preset alarm conditions, an alarm evidence package is generated and alarm information is output.

[0007] Secondly, this application provides an unknown threat detection system, comprising: The acquisition module is used to acquire network telemetry data of the network to be detected. The segmentation module is used to divide the network telemetry data into multiple time slices according to preset multiple time scales; A construction module is used to construct a connection graph based on the network telemetry data for each of the time slices, wherein the connection graph includes multiple nodes and edges connecting the nodes, the nodes are used to represent network entities, and the edges are used to represent network connection events; The extraction module is used to extract at least one graph feature based on the connectivity graph; The scoring module is used to determine anomaly scores based on the graph features; The output module is used to generate an alarm evidence package and output alarm information when the anomaly score meets the preset alarm conditions.

[0008] Thirdly, this application provides an electronic device, comprising: Processor; and, Memory for storing the executable instructions of the processor; The processor is configured to perform any of the possible methods described in the first aspect by executing the executable instructions.

[0009] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the possible methods described in the first aspect.

[0010] The unknown threat detection method and system provided in this application acquire network telemetry data of the network to be detected, divide the network telemetry data into multiple time slices according to preset multi-time scales, construct a connectivity graph based on the network telemetry data for each time slice, extract at least one graph feature based on the connectivity graph, determine anomaly scores based on the graph features, and generate an alarm evidence package and output alarm information when the anomaly score meets preset alarm conditions, thereby making the alarm verifiable and traceable, reducing false alarms caused by legitimate scanning roles and improving alarm handling efficiency. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] Figure 1 This is a schematic flowchart illustrating an unknown threat detection method according to an example embodiment of this application; Figure 2 This is a flowchart illustrating a specific implementation of S130 according to an example embodiment of this application; Figure 3 This is a schematic diagram of the structure of an unknown threat detection system according to an example embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application.

[0013] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0014] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0015] Figure 1 This is a flowchart illustrating an unknown threat detection method according to an example embodiment of this application. Figure 1 As shown, the method provided in this embodiment includes: S110. Obtain network telemetry data of the network to be tested.

[0016] Optionally, the aforementioned network telemetry data includes at least one of the following: Layer 4 traffic logging, network security device logs, and connection logs generated by network probes; The Layer 4 traffic record includes at least one of the following: five-tuple information, connection start and end time, message statistics, and transmission control protocol flag information.

[0017] In one specific implementation, network probes can be deployed at at least one of the following: the egress link, the core switching link, and the server access link of the network to be inspected. The network probes are controlled to perform parsing processing on the passing network packets and generate Layer 4 traffic records based on the parsing results. The network security device logs and the connection logs generated by the network probes are timestamped and normalized, and then written to a preset data storage device.

[0018] Optionally, for generating Layer 4 traffic records based on the parsing results, for each network connection, the source address, destination address, source port, destination port, and transport layer protocol can be extracted to generate a five-tuple of information. The connection start and end times of the network connection are recorded, and the packet statistics of the network connection are calculated. The packet statistics include at least one of the following: number of packets, number of bytes, round-trip byte ratio, and number of payload bytes. Then, the transmission control protocol (TCP) flag information corresponding to the network connection is extracted. The TCP flag information includes at least one of the following: SYN flag, ACK flag, RST flag, and FIN flag.

[0019] Optionally, the connection logs generated by the network probe include: sessionifying the network connection based on the five-tuple information, and for the sessionified network connection, outputting at least one of the following: connection establishment status, connection termination status, and failure type, to form a connection log.

[0020] Optionally, the aforementioned network security device logs include at least one of the following: Firewall logs, intrusion detection logs, intrusion prevention logs, network access control logs, and agent logs are all included. Among them, the timestamp normalization processing of network security device logs involves converting the time field of the network security device logs into a unified timestamp format with a unified time zone.

[0021] S120. Divide the network telemetry data into multiple time slices according to the preset multi-time scale.

[0022] Optionally, a first time slice set can be generated according to a first time scale, a second time slice set can be generated according to a second time scale, and a third time slice set can be generated according to a third time scale. The first time scale is smaller than the second time scale, and the second time scale is smaller than the third time scale.

[0023] Specifically, this can involve determining the window length Δt1 and overlap ratio ρ1 corresponding to the first time scale, and aligning the network telemetry data according to a preset reference time t0 based on a unified timestamp. A rolling window is executed with a window length Δt1 and a window step size set to Δt1×(1-ρ1) to generate a set of first time slices covering a preset detection period. Then, for each network connection event, based on the connection start and end times, the network connection event is written into at least one first time slice whose time range overlaps with its own, and at least one of the following is recorded in the corresponding first time slice: 5-tuple information, connection start and end times, message statistics, and transmission control protocol flag information.

[0024] Furthermore, the window length Δt2 and overlap ratio ρ2 corresponding to the second time scale are determined, and the window length Δt2 is made an integer multiple of the window length Δt1. A rolling window is performed with a window length Δt2 and a window step size set to Δt2×(1-ρ2) to generate a set of second time slices covering a preset detection period. Then, network connection events belonging to the same second time slice are aggregated according to source address, destination address, destination port, and transport layer protocol. At least one of the following is recorded within the second time slice: connection count, success status, and failure status, to obtain the aggregated network telemetry data corresponding to the second time slice.

[0025] Furthermore, the window length Δt3 and overlap ratio ρ3 corresponding to the third time scale are determined, and the window length Δt3 is made an integer multiple of the window length Δt2. A rolling window is executed with a window length Δt3 and a window step size set to Δt3×(1-ρ3) to generate a set of third time slices covering the preset detection period. Then, for network connection events spanning multiple third time slices, a time-proportion-based count update is performed within each overlapping third time slice to ensure that the connection count and message statistics satisfy the conservation constraint across multiple third time slices, and a time slice count result is generated within the third time slice for statistically analyzing the time series of the number of duplicate destination nodes.

[0026] S130. For each time slice, construct a connection map based on network telemetry data.

[0027] In this step, a connectivity graph is constructed based on network telemetry data for each time slice. The connectivity graph includes multiple nodes and edges connecting the nodes. Nodes are used to represent network entities, and edges are used to represent network connection events.

[0028] Optionally, the success and failure states of the above edges shall include at least one of the following: Handshake successful, reset failed, timeout failed, rejection failed; Among them, reset failure is used to characterize the failure type of receiving a reset message during the Transmission Control Protocol connection establishment process.

[0029] Figure 2 This is a flowchart illustrating a specific implementation of S130 according to an example embodiment of this application. For example... Figure 2 As shown, S130 includes: S131. Map the internal network host identifier to the node identifier to generate a node.

[0030] Specifically, this involves acquiring the source and destination addresses from network telemetry data and converting them into a unified internal network host identifier based on preset address normalization rules. These preset address normalization rules include at least one of private address identification, network address translation restoration, and virtualization identifier association. For each internal network host identifier, a unique node identifier is generated, and at least one of the following is recorded in the node attributes: internal network host identifier, network segment, asset role label, and hostname. When multiple alias identifiers correspond to the same internal network host identifier in the network telemetry data, these alias identifiers are merged into a single node identifier based on preset merging rules. These preset merging rules include at least one of media access control address consistency, dynamic host configuration protocol lease association, and directory service account association.

[0031] S132. Map the network connection events between the source node and the destination node as edges.

[0032] Specifically, the node identifier corresponding to the source address in the 5-tuple information is used as the source node identifier, and the node identifier corresponding to the destination address in the 5-tuple information is used as the destination node identifier. An edge key value is generated using the destination port of the network connection event and the transport layer protocol. At least one of the following is recorded on the edge: the connection start and end time, message statistics, and transmission control protocol flag information. Network connection events occurring within the same time slice and having the same source node identifier, destination node identifier, and edge key value are written into the same edge instance to form a set of edges in the connection graph.

[0033] S133. Aggregate network connection events with the same source node, destination node, and destination port to record at least one of the following on the edge: connection count, success status, and failure status.

[0034] Specifically, within each time slice, aggregation keys are generated for network connection events with the same source node identifier, destination node identifier, destination port, and transport layer protocol, and group aggregation is performed based on the aggregation keys. In each aggregated group, the connection count of the network connection event is accumulated, and the success or failure status is determined according to the transmission control protocol flag information and the connection termination status. The connection count, success status, and failure status are written as edge attributes to the edge corresponding to the aggregation key, where the failure status includes at least one of reset failure, timeout failure, and rejection failure.

[0035] S140. Based on the connectivity graph, extract at least one graph feature.

[0036] Optionally, a key port connection subgraph can be constructed for a preset set of key ports. Graph features associated with the preset set of key ports can then be extracted from the key port connection subgraph.

[0037] The preset key port set includes at least one of the following: File sharing service port, remote desktop service port, remote management service port, secure shell service port, directory service port, database service port.

[0038] It is worth noting that unknown threats in the internal network phase often do not rely on malicious payloads from a single connection, but rather manifest as anomalies in the connection relationship structure and its temporal evolution: First, in order to reduce the connection density within a single time slice, attackers will spread the probes over a longer time span, resulting in the dilution of single-granularity statistics; Secondly, in order to find available services and high-value assets, attackers will gradually expand the coverage of target nodes and increase the proportion of new target nodes. This process is not significant in a short window but shows a monotonous expansion in a longer window. Third, probing and password attempts will result in a higher failure rate, such as timeout, rejection, and reset, but failures will be further diluted by multi-port, multi-target, and multi-source collaboration. Fourth, attack tools often have a preference for a few key ports, such as SMB, RDP, SSH, and directory services, resulting in a concentrated port distribution, while legitimate business may exhibit a more stable or role-related port distribution.

[0039] It is worth noting that the common feature of the above phenomena is that abnormal signals mainly exist in the structured statistics such as the size / change of the adjacency set, the distribution of edge states, the distribution pattern of ports, and the trend over time.

[0040] To address this, the above graph features can be configured to include at least one of the following: The number of duplicate destination nodes deduplicated by the source node within the preset time slice, the proportion of new destination nodes by the source node within the preset time slice, the proportion of failed connections by the source node within the preset time slice, the concentration of destination ports of the source node, and the trend of destination node coverage expansion by the source node.

[0041] Destination port concentration includes: Calculate the concentration index based on the distribution of destination ports accessed by the source node within a preset time slice; The concentration index is used to characterize the source node's preference for a small number of destination ports.

[0042] In addition, the aforementioned trend of expanding coverage of target nodes includes: Within the time slice corresponding to the third time scale, the time series of the number of deduplicated destination nodes is statistically analyzed, and the expansion trend index is determined based on the time series. The expansion trend index is used to characterize the trend strength of the number of deduplicated destination nodes increasing over time.

[0043] For the aforementioned expansion trend index, one possible implementation is to obtain the number of duplicate destination nodes in the i-th time slice within M consecutive time slices corresponding to the third time scale, where i is an index from 1 to M. A cumulative coverage sequence is constructed based on the number of duplicate destination nodes in each time slice, where the first term of the cumulative coverage sequence equals the number of duplicate destination nodes in the first time slice, and each term from the second time slice onwards equals the cardinality of the union of the destination node sets up to the corresponding time slice, where the destination node set is the set of destination nodes visited by the source node within the corresponding time slice. A linear fit is performed on the cumulative coverage sequence to obtain the fitting slope, and the fitting determination coefficient is calculated. Then, the fitting slope and the fitting determination coefficient are normalized and weighted to obtain the expansion trend index, where the expansion trend index increases with the increase of the fitting slope and the increase of the fitting determination coefficient. When the expansion trend index meets a preset trend condition, it is determined that the destination node coverage of the source node shows an expansion trend.

[0044] It is worth noting that the above graph features are essentially a quantification of the adjacency set, edge attribute distribution, and time series trend of the connected graph within a preset time slice: the number of deduplicated destination nodes is equivalent to the cardinality of the outgoing neighbor set of the source node within that time slice, used to measure coverage. The proportion of new destination nodes reflects the strength of new coverage by differentiating and normalizing the outgoing neighbor set of the current time slice with the historical baseline set. The proportion of failed connections characterizes whether the connection state distribution is biased towards failure by counting and normalizing the edges sent by the source node according to successful and failed states. The concentration of destination ports is calculated by the concentration index based on the frequency distribution of destination ports accessed by the source node, such as equivalent indices based on entropy, Gini coefficient, or Herfindahl exponent, characterizing whether port preference is abnormally concentrated. The trend of destination node coverage expansion is characterized by forming a time series of the number of deduplicated destination nodes on a longer time scale, and using trend strength indices such as slope / growth rate, monotonicity score, or goodness of fit, which can be optionally implemented, to represent the growth pattern of coverage over time. Slow scanning, distributed probing, and lateral movement forward probing share common statistical patterns, including progressively expanding coverage, higher failure rates, and significant or phased port preferences. These characteristics can amplify diluted anomalous signals at the graph structure and temporal evolution levels, thereby enabling effective characterization and detection of unknown threats without relying on payload characteristics.

[0045] S150. Determine the anomaly score based on the graph features.

[0046] Optionally, for a single source node, a single-source anomaly score can be determined based on at least two of the following: the number of deduplicated destination nodes, the proportion of new destination nodes, the proportion of failed connections, and the concentration of destination ports. For multiple source nodes, a collaborative anomaly score can be determined based on the collaborative similarity of the multiple source nodes. An anomaly score can then be determined based on both the single-source and collaborative anomaly scores.

[0047] Specifically, for determining the single-source anomaly score, within the same preset time slice, the number of deduplicated destination nodes, the proportion of new destination nodes, the proportion of failed connections, and the concentration of destination ports corresponding to the source node are obtained. Upper limit truncation and monotonically normalized processing are applied to the number of deduplicated destination nodes and the proportion of new destination nodes, respectively. Normalization weighted by failure status is applied to the proportion of failed connections, and monotonically normalized processing is applied to the concentration of destination ports. At least two of the normalized indicators are selected as scoring components, and a weighted sum is performed according to preset weights to obtain the single-source anomaly score. The preset weights are determined based on the baseline statistical distribution of the network under test and are updated periodically.

[0048] Furthermore, the determination of the aforementioned collaborative anomaly score can be achieved within the same preset time slice, based on at least one of the following: similarity of multiple source nodes to the destination node set, similarity of destination port preferences, and similarity of failure state distribution. The set size, mean similarity within the set, and stability of the similarity within the set are calculated for the collaborative source node set. The set size, mean similarity within the set, and stability of the similarity within the set are input into a preset collaborative scoring mapping table to output the collaborative anomaly score. This collaborative scoring mapping table is used to increase the collaborative anomaly score when both the set size and the mean similarity within the set increase, and to suppress collaborative anomaly scores caused solely by occasional pairwise similarities.

[0049] For the aforementioned determination of anomaly scores, the single-source anomaly score corresponding to the source node and the collaborative anomaly score corresponding to the set of collaborating source nodes to which the source node belongs can be obtained within the same preset time slice. A confidence-weighted fusion is then performed on the single-source anomaly score and the collaborative anomaly score to obtain the anomaly score. The confidence level is determined at least based on the missing proportion of network telemetry data, the number of connections after edge aggregation, and the proportion of identifiable failure states. This confidence level is used to reduce the fusion weight of the collaborative anomaly score when the missing proportion of network telemetry data increases.

[0050] Furthermore, the collaborative similarity of the aforementioned multiple source nodes includes at least one of the following: Similarity of destination node sets among multiple source nodes, similarity of destination port preferences among multiple source nodes, and similarity of failure state distribution among multiple source nodes.

[0051] In one possible implementation, the similarity of the destination node set for the aforementioned multiple source nodes includes: Determine the intersection-union similarity (IUCN) of the destination node sets for any two source nodes; When the intersection-union similarity meets the preset similarity condition, the two source nodes are determined to be cooperative source nodes.

[0052] Specifically, within the same preset time slice, the first destination node set corresponding to the first source node and the second destination node set corresponding to the second source node can be obtained respectively. Both the first destination node set and the second destination node set are obtained by deduplicating the destination nodes pointed to by the outgoing edges of the corresponding source nodes in the connection graph.

[0053] The first and second destination node sets are subjected to set standardization processing. The set standardization processing includes at least deduplication processing based on node identifiers and elimination processing based on preset time validity conditions. The preset time validity conditions are used to eliminate destination nodes corresponding to network connection events that did not occur within the preset time slice.

[0054] Determine the cardinality of the intersection of the first and second target node sets, and determine the cardinality of the union of the first and second target node sets. The ratio of the intersection cardinality to the union cardinality is defined as the intersection-union similarity ratio (OCR). When the union cardinality is zero, the OCR is set to zero.

[0055] In another possible implementation, the similarity of the destination node sets of the above multiple source nodes includes: A hash digest is generated for the set of source nodes and destination nodes. Then, an approximate similarity search is performed based on the hash digest generation result to determine the set of collaborative source nodes that meet the preset similarity conditions.

[0056] Specifically, within the same preset time slice, the set of destination nodes corresponding to the source node can be obtained. The set of destination nodes is obtained by deduplicating the destination nodes pointed to by the outgoing edges of the source node in the connection graph. Normalization processing is performed on the node identifiers of each destination node in the set of destination nodes to obtain a normalized node identifier sequence. The normalization processing includes at least unifying the node identifiers to a preset encoding format and removing null node identifiers. The normalized node identifier sequence is sorted according to a preset sorting rule to form an ordered node identifier list. The preset sorting rule includes at least one of lexicographical sorting or numerical sorting. The ordered node identifier list is concatenated using a preset delimiter to generate a set representation string. A preset hash algorithm is performed on the set representation string to obtain a hash digest generation result. The preset hash algorithm includes at least one of SHA-256, SHA-1, or MD5.

[0057] Then, for each source node, the target node set is divided into multiple subsets, and a preset hash algorithm is performed on each subset to generate multiple sub-digests, thus forming a multi-sub-digest signature for the source node. The multi-sub-digest signatures of each source node are written into a Local Sensitive Hash Index (LSI) structure so that source nodes with the same sub-digests are assigned to the same candidate bucket. For the target source node, a set of candidate source nodes located in the same candidate bucket as the target source node is retrieved from the LSI structure. The intersection-union (IU) similarity between each candidate source node in the candidate source node set and the target source node is determined. When the IU similarity meets a preset similarity condition, the candidate source node is determined as a cooperative source node that meets the preset similarity condition.

[0058] Within the same preset time slice, approximate similarity searches are performed pairwise on each source node to obtain collaborative source node pairs that meet preset similarity conditions. Each collaborative source node pair is constructed as an undirected collaborative relationship graph, where graph nodes are source nodes and graph edges represent collaborative relationships. Connectivity component partitioning is performed on the undirected collaborative relationship graph to identify source nodes belonging to the same connected component as the same collaborative source node set. The collaborative source node set is output for determining the collaborative similarity of multiple source nodes.

[0059] It's worth noting that in the detection of unknown threats based on network telemetry data, anomaly scores are often calculated using connection graphs and graph features to detect abnormal behaviors such as internal network scanning, lateral movement probing, and weak password attempts. However, in real networks, legitimate high-frequency connection sources such as maintenance jump servers, centralized management platforms, asset inventory and vulnerability assessment systems also access a large number of destination nodes and critical ports in a short period of time, possibly accompanied by a certain proportion of failed connections. Thus, under a detection framework that relies solely on graph features such as connection scale, failure rate, port concentration, and coverage expansion trends, these connections exhibit statistical characteristics similar to those of unknown threats, leading to false alarms, alarm fatigue, and a waste of security operation resources.

[0060] To address this, after identifying the abnormal scores, we can further obtain the asset role labels corresponding to each node in the network to be detected. Then, based on the asset role labels, we can perform threshold adjustment or suppression processing on the abnormal scores to obtain the adjusted abnormal scores. The asset role labels include at least permitted scanning role labels and non-permitted scanning role labels.

[0061] Optionally, the above-mentioned allowed character tags include at least one of the following: Role tags for asset inventory, vulnerability assessment, monitoring and management, operations and maintenance platform, and centralized management.

[0062] In addition, the threshold adjustment or suppression processing for abnormal scores based on asset role labels mentioned above includes: When the source node has a "scan allowed" role label, increase the threshold corresponding to the preset alarm condition or convert the alarm type to an audit alarm; When the source node has a non-allowed scanning role label, lower the threshold corresponding to the preset alarm condition or increase the weight of the abnormal score.

[0063] Through the above steps, the same anomaly score can be mapped to different decision boundaries under different asset roles: First, nodes are assigned permitted or prohibited scanning role labels based on the asset management system, CMDB, operation and maintenance platform, or pre-configured policies; then, using the asset role label as a control variable, the threshold, anomaly score weight, or alarm type of the preset alarm conditions are conditionally adjusted, which is equivalent to grouping and calibrating / stratifying the detection distribution, so that the score is aligned with its normal baseline within each role group. Through this role-based stratified threshold shifting and weight recalibration, high-score noise from legitimate roles can be suppressed, while the equivalent anomaly characteristic signals on prohibited roles can be strengthened, thereby achieving a balance between reducing false alarms and maintaining sensitivity.

[0064] S160. When the abnormal score meets the preset alarm conditions, generate an alarm evidence package and output alarm information.

[0065] Optionally, within the same preset time slice, the corresponding anomaly score and the corresponding adjusted anomaly score are obtained for each source node set. Based on the asset role tags of the source node set, an alarm threshold corresponding to the preset alarm condition is determined from a preset threshold configuration table. This preset threshold configuration table at least associates with an alarm type identifier, asset role tag, time scale identifier, and key port identifier. The adjusted anomaly score is compared with the alarm threshold. If the adjusted anomaly score is greater than or equal to the alarm threshold, the anomaly score is determined to meet the preset alarm condition.

[0066] The set of source nodes that meet the preset alarm conditions is subjected to deduplication and merging and alarm suppression processing. The alarm suppression processing includes at least repeated alarm suppression based on a preset silence duration and alarm suppression based on a preset whitelist.

[0067] After determining that the abnormal score meets the preset alarm conditions, an alarm evidence package is generated, which includes the source node set, destination node set, destination port distribution, failure mode fingerprint, and coverage expansion trend within the corresponding time slice of the third time scale. Alarm information including alarm type, alarm level, trigger time slice identifier, and alarm evidence package index is output.

[0068] Specifically, the aforementioned alarm evidence package includes: Determine the set of source nodes that trigger the preset alarm conditions; Determine the set of destination nodes associated with the set of source nodes and the distribution of destination ports; Determine the coverage expansion trend of the source node set within the corresponding time slice of the third time scale; Determine the failure mode fingerprint corresponding to the source node set; An alarm evidence package is generated based on the above determination results.

[0069] Specifically, for the set of source nodes that trigger the preset alarm conditions, the adjusted anomaly score and the alarm threshold corresponding to the preset alarm conditions can be obtained for each source node within the same preset time slice. Source nodes whose adjusted anomaly scores are greater than or equal to the alarm thresholds are identified as candidate source nodes. The candidate source nodes are deduplicated according to their source node identifiers, and repeatedly triggered candidate source nodes are merged based on a preset silence duration to obtain the set of source nodes that trigger the preset alarm conditions.

[0070] To determine the destination node set and destination port distribution associated with the source node set, the following steps can be taken: For each source node in the source node set, within the time slice when the preset alarm condition is triggered, obtain the set of outgoing edges originating from the source node in the connection graph, and extract the corresponding destination node identifier and destination port identifier from the outgoing edge set. Deduplication is then performed on the extracted destination node identifiers to obtain the destination node set. Group counting and percentage statistics are then performed on the extracted destination port identifiers to obtain the destination port distribution. The group counting is weighted based on at least one of the following: connection count, success status, and failure status.

[0071] To determine the coverage expansion trend of the source node set within the corresponding time slice of the third time scale, the following steps can be taken: For each source node in the source node set, obtain the time series of the number of duplicate target nodes within multiple consecutive time slices corresponding to the third time scale. Perform smoothing and denoising processing on the time series of the number of duplicate target nodes to obtain a smoothed time series. Based on the smoothed time series, determine the expansion trend index, and identify source nodes whose expansion trend index meets preset trend conditions as coverage expansion source nodes. Use the coverage expansion source nodes and their corresponding expansion trend index as the coverage expansion trend of the source node set within the corresponding time slice of the third time scale.

[0072] To determine the failure mode fingerprint corresponding to the aforementioned set of source nodes, the following steps can be taken: For each source node in the set, within the time slice that triggers a preset alarm condition, obtain the failure state and corresponding connection count from the edge attributes associated with the source node in the connection graph. Merge the failure states according to a preset set of failure state types, and determine the failure state distribution based on the connection count of each failure state. Jointly encode the failure state distribution and the destination port distribution to generate the failure mode fingerprint of the source node. The joint encoding at least includes deduplicating and sorting the combination of failure state type and destination port identifier, and generating a fixed-length digest.

[0073] Then, an alarm event identifier is generated for each alarm event, and the alarm event identifier is associated with the trigger time slice identifier, alarm type, and alarm level. The source node set, destination node set, destination port distribution, coverage expansion trend, and failure mode fingerprint are recorded in the alarm evidence packet. Furthermore, the asset role label corresponding to the source node set and the network telemetry data index for traceability are recorded. The alarm evidence packet is stored in a structured serialized format, and alarm information containing the alarm event identifier and the storage location of the alarm evidence packet is output.

[0074] Finally, the visualization results of the alarm evidence package are output on the security operations platform. The visualization results include at least a connection graph view and a key port connection subgraph view.

[0075] In this embodiment, network telemetry data of the network to be detected is acquired and divided into multiple time slices according to a preset multi-time scale. For each time slice, a connectivity graph is constructed based on the network telemetry data. Then, based on the connectivity graph, at least one graph feature is extracted, and an anomaly score is determined according to the graph feature. When the anomaly score meets the preset alarm conditions, an alarm evidence package is generated and an alarm message is output, thereby making the alarm verifiable and traceable, reducing false alarms caused by legitimate scanning roles and improving alarm handling efficiency.

[0076] Specifically, the above embodiments aggregate and model network connection events across multiple time scales, allowing low-frequency events to accumulate over long time scales to form distinguishable coverage features. A connection graph is constructed within each time slice, transforming discrete connection attempts into a computable graph structure. Furthermore, a key port connection subgraph is constructed from the key port set, and port preference-related features are extracted. A failure mode fingerprint is formed by combining the failure state distribution. The set of collaborative source nodes is identified through similarity and approximate similarity retrieval of the target node set, achieving structural anomaly characterization of multi-source collaboration, sparse persistent edges, and coverage expansion. Finally, an anomaly score is obtained by fusing single-source anomaly scores and collaborative anomaly scores. A structured evidence package is output when preset alarm conditions are met, elevating the detection result from single-point threshold triggering to an interpretable alarm constrained by both graph structure and temporal evolution.

[0077] Figure 3 This is a schematic diagram illustrating the structure of an unknown threat detection system according to an example embodiment of this application. Figure 3 As shown, the unknown threat detection system 300 provided in this embodiment includes: The acquisition module 310 is used to acquire network telemetry data of the network to be detected. The segmentation module 320 is used to divide the network telemetry data into multiple time slices according to a preset multi-time scale; The construction module 330 is used to construct a connection graph based on the network telemetry data for each of the time slices, wherein the connection graph includes multiple nodes and edges connecting the nodes, the nodes are used to represent network entities, and the edges are used to represent network connection events. Extraction module 340 is used to extract at least one graph feature based on the connectivity graph; Scoring module 350 is used to determine anomaly scores based on the graph features; The output module 360 ​​is used to generate an alarm evidence package and output alarm information when the anomaly score meets the preset alarm conditions.

[0078] Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application. For example... Figure 4 As shown, the electronic device 400 provided in this embodiment includes: a processor 401 and a memory 402; wherein: Memory 402 is used to store computer programs, and the memory may also be flash memory.

[0079] Processor 401 is used to execute the execution instructions stored in the memory to implement the various steps in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0080] Alternatively, the memory 402 can be either standalone or integrated with the processor 401.

[0081] When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include: Bus 403 is used to connect the memory 402 and the processor 401.

[0082] This embodiment also provides a readable storage medium storing a computer program, which, when executed by at least one processor of an electronic device, enables the electronic device to perform the methods provided in the various embodiments described above.

[0083] This embodiment also provides a program product including a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the methods provided in the various embodiments described above.

[0084] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0085] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting unknown threats, characterized in that, include: Acquire network telemetry data of the network to be tested; The network telemetry data is divided into multiple time slices according to a preset multi-time scale; For each of the aforementioned time slices, a connectivity graph is constructed based on the network telemetry data. The connectivity graph includes multiple nodes and edges connecting the nodes. The nodes are used to represent network entities, and the edges are used to represent network connection events. Based on the connectivity graph, at least one graph feature is extracted; Anomaly scores are determined based on the graph features; When the anomaly score meets the preset alarm conditions, an alarm evidence package is generated and alarm information is output.

2. The method according to claim 1, characterized in that, The network telemetry data includes at least one of the following: Layer 4 traffic logging, network security device logs, and connection logs generated by network probes; The four-layer traffic record includes at least one of the following: five-tuple information, connection start and end time, message statistics, and transmission control protocol flag information.

3. The method according to claim 1, characterized in that, The process of dividing the network telemetry data into multiple time slices according to a preset multi-time scale includes: Generate a set of first-time slices according to the first time scale; Generate a second time slice set according to the second time scale; Generate a set of third time slices according to the third time scale; Wherein, the first time scale is smaller than the second time scale, and the second time scale is smaller than the third time scale.

4. The method according to claim 1, characterized in that, The construction of the connectivity graph based on the network telemetry data includes: Map the internal network host identifier to the node identifier to generate the node; Map the network connection events between the source node and the destination node to the edges; Network connection events with the same source node, destination node, and destination port are aggregated to record at least one of the following on the edge: connection count, success status, and failure status.

5. The method according to claim 1, characterized in that, The success and failure states of the edge include at least one of the following: Handshake successful, reset failed, timeout failed, rejection failed; The reset failure is used to characterize the failure type of receiving a reset message during the Transmission Control Protocol connection establishment process.

6. The method according to claim 1, characterized in that, The step of extracting at least one graph feature based on the connectivity graph includes: For a predefined set of key ports, construct a key port connection subgraph; Extract graph features associated with the preset set of key ports from the key port connection subgraph.

7. The method according to claim 6, characterized in that, The preset key port set includes at least one of the following: File sharing service port, remote desktop service port, remote management service port, secure shell service port, directory service port, database service port.

8. The method according to claim 1, characterized in that, The step of determining the anomaly score based on the graph features includes: For a single source node, a single-source anomaly score is determined based on at least two of the following: the number of deduplicated destination nodes, the proportion of new destination nodes, the proportion of failed connections, and the concentration of destination ports. For multiple source nodes, a collaborative anomaly score is determined based on the collaborative similarity of the multiple source nodes; The anomaly score is determined based on the single-source anomaly score and the collaborative anomaly score.

9. The method according to claim 1, characterized in that, The generated alarm evidence package includes: Determine the set of source nodes that trigger the preset alarm conditions; Determine the set of destination nodes and the distribution of destination ports associated with the set of source nodes; Determine the coverage expansion trend of the source node set within the corresponding time slice of the third time scale; Determine the failure mode fingerprint corresponding to the set of source nodes; The alarm evidence package is generated based on the above determination results.

10. An unknown threat detection system, characterized in that, include: The acquisition module is used to acquire network telemetry data of the network to be detected. The segmentation module is used to divide the network telemetry data into multiple time slices according to preset multiple time scales; A construction module is used to construct a connection graph based on the network telemetry data for each of the time slices, wherein the connection graph includes multiple nodes and edges connecting the nodes, the nodes are used to represent network entities, and the edges are used to represent network connection events; The extraction module is used to extract at least one graph feature based on the connectivity graph; The scoring module is used to determine anomaly scores based on the graph features; The output module is used to generate an alarm evidence package and output alarm information when the anomaly score meets the preset alarm conditions.

Images