Network evidence obtaining and attack chain reconstruction method based on threat alarm

Abstract

The invention discloses a network evidence obtaining and attack chain reconstruction method based on threat alarm, and belongs to the field of network security. Aiming at the problems of massive alarm fragmentation, strong attack chain concealment, low evidence obtaining efficiency and the like existing in a traditional network evidence obtaining method, the invention provides a dynamic reconstruction model of multi-dimensional alarm aggregation and probabilistic reasoning. Threat alarms are aggregated through an improved DBSCAN algorithm, an attack behavior graph is constructed, and attack chain probabilistic reasoning is carried out in combination with a time sequence Petr i network and an HMM. Experiments prove that the alarm aggregation compression rate reaches 92.3%, the attack chain reconstruction accuracy rate reaches 88.6%, the time consumed by single analysis is less than 3 seconds, and the network attack tracing and evidence obtaining efficiency is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a network forensics and attack chain reconstruction method based on threat alerts, which is used for fusing multi-source threat information, dynamically tracking attack behaviors and reconstructing complete attack chains to support network attack tracing and security protection decision-making. BACKGROUND

[0002] With the evolution of network attack technology, advanced persistent threats (APTs) and zero-day exploit pose a serious challenge to network security. Traditional network forensics methods rely on log analysis and human experience, which have the following problems:

[0003] Massive threat alert fragmentation: the alerts generated by security devices (such as IDS, firewall) are large in number and isolated, making it difficult to correlate and analyze;

[0004] Enhanced concealment of attack chain: attackers use multi-stage penetration strategies (such as the tactical stages in the ATT&CK framework), making it difficult to track cross-system and cross-period behaviors;

[0005] Low forensics efficiency: existing methods (such as rule-based feature matching) have high false positive rates and lack global reasoning ability for attack intent.

[0006] In the prior art, the STIX / TAXII framework only realizes static intelligence exchange and cannot dynamically correlate the time sequence characteristics of real-time alerts; the method based on graph neural network (GNN) lacks attack chain full-stage time sequence modeling capability; the traditional clustering algorithm (such as DBSCAN) uses static weights and cannot adapt to the time sensitivity differences of different attack types; the HMM application is limited to single-stage analysis and lacks dynamic linkage with attack graphs. Therefore, there is an urgent need for an innovative method that fuses multi-source threat alerts and has time sequence reasoning capability.

[0007] Terminology and Abbreviation Full Name

[0008] APT: Advanced Persistent Threat

[0009] IDS: Intrusion Detection System

[0010] STIX / TAXII: Structured Threat Information eXpression / Trusted Automated

[0011] Exchange Indicator Information

[0012] eXchange of Indicator Information

[0013] GNN: Graph Neural Network

[0014] DBSCAN: Density-Based Spatial Clustering of Applications with Noise

[0015] Clustering of Applications with Noise)

[0016] HMM: Hidden Markov Model

[0017] LSTM: Long Short-Term Memory

[0018] ATT&CK: Adversarial Tactics, Techniques, and Common Knowledge

[0019] Common Knowledge) CVSS: Common Vulnerability Scoring System CVE: Common Vulnerabilities and Exposures CWE: Common Weakness Enumeration PTP: Precision Time Protocol SUMMARY

[0020] The present application proposes a network forensics and attack chain reconstruction method based on threat alert, the core of which is a multi-dimensional alert aggregation and probability reasoning attack chain dynamic reconstruction model, which is realized by the following steps:

[0021] Multi-dimensional aggregation of threat alerts: using an improved DBSCAN algorithm, combining dynamic weight adjustment (entropy weight method initialization + LSTM optimization) and time decay factor of attack type adaptation, clustering the original alerts to generate alert clusters, outputting cluster center features, confidence and evidence pointers;

[0022] Attack behavior graph construction: Alarm clusters are mapped to graph nodes (including core features and additional information). Node weights are calculated based on CVSS scores and frequency. Edges are constructed and weights are calculated based on temporal density and ATT&CK tactical transfer probability. The graph is then optimized through pruning and verification.

[0023] Attack chain probabilistic reasoning: Temporal Petri nets and HMMs are used for joint modeling. Hidden states in the attack phase and observed states of alarm clusters are defined. Weight coefficients are trained by LSTM and the Viterbi algorithm is used to solve for the most likely attack path, supporting security situation awareness, attack tracing and policy adjustment.

[0024] Technical solution

[0025] To address the problems of fragmented threat alerts, difficulty in attack chain tracing, and low efficiency in traditional network forensics methods, this invention aims to provide a method that integrates multi-source alerts and possesses temporal reasoning capabilities. This method enables dynamic reconstruction and precise source tracing of attack chains, improving the efficiency and accuracy of network attack forensics and providing decision support for network security protection. The specific implementation process is as follows:

[0026] Step 1: Multi-dimensional aggregation of threat alerts

[0027] 1.1 Input Features and Data Model

[0028] Input: Original threat alert set A = {a1, a2, ..., a...} n Each alarm a i It has the following attributes:

[0029] timestamp t i UTC time is accurate to the microsecond level and is calibrated using a hardware clock synchronization protocol (such as the PTP protocol) to ensure time consistency across nodes.

[0030] Source IP It supports both IPv4 and IPv6 addresses, and covers virtual network address spaces, such as container IPs and cloud server intranet IPs.

[0031] Destination port It is associated with transport layer protocols (such as TCP and UDP) and application layer services (such as port 80 corresponding to the HTTP protocol).

[0032] Attack type tag l i Follow the CVE / CWE standard classification system, such as "SQL Injection (CWE-89)" and "Man-in-the-Middle Attack (CVE-2024-1234)".

[0033] Additional metadata: Contains device type (such as server, terminal, industrial gateway), operating system (such as Windows, Linux, Android), sensor node ID (used to specify the source of data collection).

[0034] Data preprocessing:

[0035] Regularization method is used for parsing unstructured fields (such as attack payload summary) to extract key feature vectors.

[0036] Use Bloom filter to filter repeated alarms, and control the false positive rate to be less than or equal to 0.1%.

[0037] 1.2 Improved DBSCAN algorithm design

[0038] Core innovation: Introduce dynamic weight adjustment mechanism + time decay factor collaborative design, break through the static weight defect of traditional clustering algorithm. Specifically:

[0039] Dynamic weight adjustment: Through the two-stage mechanism of entropy weight method initialization and LSTM dynamic optimization, realize the adaptive update of weight coefficient (ω t +ω s +ω d +ω l =1), solve the problem that traditional fixed weight (such as equal weight Euclidean distance) cannot reflect the dynamic change of attack features. For example, automatically increase the source IP association weight (ω s ) for cross-platform attack scenarios, and automatically reduce the port weight (ω d ) for high-frequency port attacks.

[0040] Time decay factor: Combine with attack type to dynamically adjust decay parameter α (such as APT attack α = 0.01 / minute, ransomware α = 0.1 / minute), so that recent alarms contribute significantly more to clustering results than historical data, effectively distinguishing between persistent attacks and occasional behavior.

[0041] 1.2.1 Distance metric formula enhancement

[0042] 1.2.1.1 Define the weighted dynamic distance function between alarms

[0043] D(a i ,a j )=ω t ·|t i -t j |·exp(-α|t i -t0|)+ω s ·Sim s +ω d ·Sim d +ω l• Sim l

[0044] Parameter Description:

[0045] Time decay term: exp(-α|t i -t0|) is the time decay factor, where t0 is the current system time, and α is dynamically adjusted according to the attack type. For example, for APT attacks, α takes the value of 0.01 / minute; for ransomware attacks, α takes the value of 0.1 / minute.

[0046] Weight coefficient:

[0047] ω t +ω s +ω d +ω l = 1, which is adaptively calculated by the following two-stage mechanism:

[0048] Entropy weight method initialization: dynamically assign initial weights according to the information entropy of alarm attributes. For example, for frequently used ports (such as 443), their corresponding ω d will automatically decrease.

[0049] LSTM dynamic optimization: introduce long short-term memory network (LSTM) to learn weight adjustment strategies based on historical attack patterns. For example, when cross-platform attacks are detected, ω s (source IP association weight) will automatically increase.

[0050] Similarity function:

[0051] Sim s : Source IP address similarity, using the edit distance combined with CIDR mask matching method. For example, if two IP addresses belong to the same subnet (such as 192.168.1.0 / 24), their similarity is 0.9.

[0052] Sim d : Destination port semantic similarity, based on port-service mapping table. For example, 80 port and 443 port both belong to web service, their similarity is 0.7.

[0053] Sim l : Attack type label cosine similarity, based on Word2Vec trained CVE vector space model.

[0054] 1.2.1.2 Distance metric formula derivation process

[0055] Formula design background: traditional Euclidean distance cannot integrate time decay and semantic association, so linear combination of time decay term, weight coefficient and similarity function is introduced.

[0056] The derivation process is as follows:

[0057] (1) Time decay term: exponential decay model exp(-α|t i -t0|) is adopted, where α is positively correlated with the time sensitivity of the attack type (e.g., ransomware attacks have high frequency and require rapid decay of historical data, so α = 0.1 / minute).

[0058] (2) Weight normalization: ω t +ω s +ω d +ω l =1 ensures the additivity of the contribution of each dimension and avoids the influence of numerical scale difference on clustering results.

[0059] (3) Semantic similarity modeling:

[0060] Source IP similarity Sim s Combined with edit distance and CIDR mask, for example, IP addresses 192.168.1.10 and 192.168.1.20 belong to the same subnet (mask 24 bits), and the edit distance is 10, so

[0061] Attack type label similarity Sim l Based on the CVE vector trained by Word2Vec, the cosine similarity is calculated, such as the vector cosine value of "SQL injection (CWE-89)" and "code injection (CWE-94)" is 0.85.

[0062] 1.2.1.3 Improved DBSCAN pseudo code is as follows:

[0063] Input: original alarm set A, time decay parameter α, distance threshold ∈, minimum point number MinPts

[0064] Output: alarm cluster set C

[0065] Preprocessing:

[0066] 1. Perform Bloom filter deduplication on A, and normalize and parse unstructured fields

[0067] 2. Initialize weight vector ω = (ω t ,ω s ,ω d ,ω l ) via entropy weight method

[0068] Training phase:

[0069] 3. Train the LSTM model using historical alarm sequences:

[0070] Input: Sequence of alert features within a sliding window [a_{t-n},...,a_t]

[0071] Output: Weight adjustment factor Δω = LSTM(ω_prev, sequence of attack types)

[0072] Loss function: Mean Squared Error (MSE), optimization goal is to minimize intra-cluster distance variance

[0073] Clustering phase:

[0074] 4. FOR each alert a_i in A:

[0075]

[0076] 5. Construct density reachable graph based on D, identify core objects and boundary objects

[0077] 6. Perform breadth-first search on each core object, generate alert clusters C

[0078] 7. Validate isolated alerts through Byzantine fault tolerance protocol, exclude false positives

[0079] Key difference from traditional DBSCAN: traditional algorithm uses static time window (e.g. aggregate within 1 hour) and fixed weights (e.g. equal weight for each attribute), while this formula realizes joint modeling of "temporal proximity + behavioral semantic correlation" through dynamic weights and exponential decay term. For example, recent alerts (|t s -t0|<5 minutes) with similar source IP (Sim d =0.9) within the same subnet or similar web service port (Sim i =0.7) will be preferentially aggregated, while low-similarity alerts across platforms and time periods are effectively isolated.

[0080] 1.2.2 Density reachability optimization

[0081] Core object definition: for an alert a i , if its ∈-neighborhood contains at least MinPts valid alerts (valid defined as D(a i ,a j )≤∈×(1+δ), where δ is a dynamic tolerance factor, range is 0.1-0.3), then a i is determined as a core object.

[0082] Abnormal alert filtering: for isolated alerts that cannot be assigned to any cluster, cross-node verification through Byzantine fault tolerance protocol is performed to exclude false positives caused by sensor failures.

[0083] 1.2.3 Comparative experiment: improved DBSCAN vs traditional clustering algorithms

[0084] To verify the non-obviousness of the improved DBSCAN, comparative experiments were conducted in the simulation environment in Section 4.1, and the results are as follows:

[0085]

[0086] Key difference analysis:

[0087] Compression rate improvement: The improved DBSCAN aggregates similar attacks (such as 5000 port scans → 1 cluster) through dynamic weight, while K-means splits the same attack into multiple clusters due to its dependence on static distance threshold.

[0088] Misaggregation rate reduction: The original DBSCAN cannot distinguish the time sensitivity of APT and ransomware, and misaggregates APT alerts across days with short-time ransom attacks (misaggregation rate 22%), while the improved version avoids this problem through dynamic adjustment of α.

[0089] Efficiency advantage: The time complexity of the improved DBSCAN is O(n log n), which is better than the O(n^2) of K-means, reducing the time consumption by 57% in large-scale data.

[0090] 1.3 Output results

[0091] Cluster output: The aggregated threat alert cluster C = {c1, c2, …, c m} contains the following information:

[0092] Cluster center feature vector Where is the weighted average time (weight

[0093] Attack chain confidence This confidence reflects the consistency of alerts within the cluster.

[0094] Correlation evidence chain pointer set {p1, p2, …, p k}, these pointers point to the original evidence nodes in the storage structure.

[0095] 2. Step 2: Attack behavior graph construction

[0096] 2.1 Graph node mapping

[0097] Input: Threat alert cluster set C = {c1, c2, …, c m} obtained after multi-dimensional aggregation in step 1.

[0098] Mapping rule: Map each alert cluster c i to a node v iV. The information contained in the node not only has the core features of the alert cluster, but also has additional information related to it. Specifically as follows:

[0099] Core features: inherit the central feature vector of the alert cluster where \bar{t} is the weighted average time, is the source IP-related feature representation, is the destination port-related feature, is the attack type label.

[0100] Additional information: contains the number of alerts in the alert cluster, which can reflect the scale of the attack behavior; and the aggregation time range of the alert cluster, which helps to analyze the duration of the attack behavior.

[0101] 2.2 Node weight calculation

[0102] Node weight w v determined by the threat level (CVSS score) and frequency of the alert cluster. The specific calculation method is as follows:

[0103] CVSS score: score the attack type in the alert cluster according to the Common Vulnerability Scoring System (CVSS). The CVSS score takes into account multiple factors such as attack availability, impact range, etc., to get a score value S of 0-10 cvss . For example, for a high-risk remote code execution vulnerability, its CVSS score may be close to 10 points; while for some low-risk information disclosure vulnerabilities, the score may be around 2-3 points.

[0104] Frequency statistics: statistics of the frequency F of the alert cluster within a certain time window. The higher the frequency, the more frequent the attack behavior, and the greater the potential threat.

[0105] Node weight formula:

[0106]

[0107] where α is a weight coefficient, and the value range is 0 < α < 1, which can be adjusted according to actual conditions. For example, when paying more attention to the severity of the attack, α can take a larger value; when paying more attention to the frequency of the attack, α can take a smaller value. F max is the maximum value of the frequency of all alert clusters, used for normalization of the frequency.

[0108] 2.3 Edge construction and edge weight calculation

[0109] Edge construction rule: edge E is used to represent the time sequence and behavior dependency relationship between nodes. If node v i The time of the corresponding alert cluster is earlier than the time of node v jIf two nodes have corresponding alert clusters and the attack types of the two alert clusters have certain behavioral correlation, then a directed edge v is constructed between the two nodes i →v j .

[0110] Edge weight calculation: the edge weight w e Based on the time sequence density and the tactical phase correlation, the calculation formula is as follows:

[0111]

[0112] Time sequence density: Used to measure the closeness of the time of the two nodes corresponding to the alert cluster. |t j -t i | is the weighted average time difference of the two alert clusters. The smaller the time difference, the larger the value, indicating that the two attack behaviors are closer in time and have stronger correlation. For example, when |t j -t i | = 0, indicates that the two attack behaviors occur almost simultaneously; when |t j -t i | is large, the value tends to 0, indicating that the two attack behaviors have weak correlation in time.

[0113] Tactical phase correlation: Match(l i ,l j ) is the tactical phase transfer probability of the attack type in the ATT&CK framework, obtained by a pre-trained model. The ATT&CK framework is a comprehensive network attack behavior knowledge base, which divides attack behaviors into multiple tactical phases, such as initial access, execution, and persistence. The pre-trained model will learn the transfer probability between different attack types in these tactical phases based on a large amount of historical attack data. For example, the probability of transferring from the "initial access" tactical phase to the "execution" tactical phase may be higher, while the probability of transferring directly from "initial access" to "lateral movement" tactical phase may be lower.

[0114] 2.4 Optimization and verification after graph construction

[0115] Graph optimization:

[0116] Pruning operation: for edges with too small edge weights, it is considered that the correlation between the nodes represented by these edges is weak, and these edges can be deleted to simplify the graph structure and reduce the computational complexity.

[0117] Node merging: if the feature similarity of some nodes is very high and the edge weight between them is large, these nodes can be merged into one node to improve the abstraction level of the graph.

[0118] Graph validation:

[0119] Consistency verification: Check if the information of nodes and edges in the graph is consistent with the original alert data, ensuring that no information is lost or incorrect.

[0120] Reasonableness verification: Combined with the experience and domain knowledge of security experts, verify whether the attack behavior relationship represented by the graph is reasonable. For example, check if there is an unreasonable tactical phase transfer.

[0121] 3. Step 3: Attack chain probability reasoning

[0122] 3.1 Attack chain modeling based on HMM

[0123] Adopt the mechanism of "combined modeling of time Petri net + HMM", break through the limitation of traditional HMM which can only handle linear state transition. Time Petri net is used to explicitly model the timing constraints of attack phases (such as "lateral movement" must occur after "initial access" and the interval is < 24 hours), and HMM learns the state transition probability through the edge weight of the attack behavior graph. The combination of the two can accurately depict the "time sequence + tactical dependency" of the attack chain. For example, for the transition of "privilege escalation → data exfiltration", the time difference needs to be < 30 minutes according to the time Petri net, and HMM calculates the probability through the graph edge weight (reflecting the frequency of this transition in historical data), solving the problem that traditional methods cannot distinguish between "legal privilege change" and "attack privilege elevation".

[0124] 3.1.1 Hidden state definition

[0125] Hidden state Q = {q1, q2, …, q N} represents different stages in the attack process. These stages are divided according to common network attack patterns and processes, as follows:

[0126] Initial access: Attackers attempt to gain access to the target system through various means (such as exploiting vulnerabilities, social engineering, etc.). For example, attackers may send phishing emails to induce users to click on links, thereby planting malicious software on the user's device and gaining initial access to the target network.

[0127] Execution: After gaining initial access, attackers will execute malicious code on the target system to further expand their control range. This may include executing scripts, installing backdoor programs, etc.

[0128] Persistence: In order to ensure long-term existence on the target system, attackers will take measures to ensure that malicious software or backdoor programs can still run after system restart. For example, modify the system startup items and add malicious programs to them.

[0129] Lateral movement: Once the attacker has a foothold in the target network, they will attempt to move between different systems and devices to gain more access and data. This can involve cracking other user's account passwords, exploiting vulnerabilities within the internal network, etc.

[0130] Privilege escalation: The attacker will attempt to escalate their privileges on the target system to be able to perform more sensitive operations. For example, by exploiting a vulnerability to elevate a normal user's privileges to an administrator's privileges.

[0131] Data exfiltration: After gaining enough access and data, the attacker will exfiltrate sensitive data from the target system and transmit it to an external server. This can be done through encrypted tunnels, covert channels, etc.

[0132] Difference from traditional HMM: Traditional methods only divide states based on a single alert feature (such as attack type), while this scheme expands the state to a multi-dimensional feature space containing "time window + IP association + port service" by combining the structured constraints of time Petri nets, improving state discrimination.

[0133] 3.1.2 Observation state definition

[0134] The observation state O = {o1, o2, …, o T} is composed of the features of the alert cluster. Each alert cluster is obtained through multi-dimensional aggregation in step 1 and contains information from multiple original threat alerts. The features of the alert cluster include but are not limited to timestamp, source IP, destination port, attack type label, etc. For example, an alert cluster may show that multiple attacks with a specific attack type were launched from a certain source IP address to a specific destination port at a certain time.

[0135] 3.1.3 LSTM network structure and training details

[0136] Network architecture:

[0137] Number of layers: 2 layers of bidirectional LSTM + 1 layer of fully connected layer;

[0138] Number of neurons: 128 neurons per layer, Dropout rate 0.2;

[0139] Input dimension: alert feature vector (time difference, source IP edit distance, port service category, attack type label Embedding);

[0140] Output dimension: 4-dimensional weight adjustment coefficient Δω = (Δω t , Δω s , Δω d , Δω l ).

[0141] Training data:

[0142] Source: Real attack logs in the past 12 months, containing 100,000 labeled attack chain data;

[0143] Preprocessing: Group the alert sequence by attack stage (such as initial access -> execution) to generate a time series feature sequence.

[0144] Loss function:

[0145]

[0146] Where λ = 0.01 is the L1 regularization coefficient to avoid overfitting.

[0147] 3.2 Viterbi algorithm to solve the most likely attack path

[0148] Viterbi algorithm is a dynamic programming algorithm used to find the most likely hidden state sequence in HMM. Our goal is to find the most likely attack stage sequence (q1, q2, …, qT) from the observed alert cluster features O, i.e. the most likely attack path. N

[0149]

[0150] 3.2.1 State transition probability P(q t |q t-1 )

[0151] State transition probability P(q t |q t-1 ) describes the probability of transitioning from attack stage q t-1 to attack stage q t . We learn these probabilities from the edge weights of the attack behavior graph. In the attack behavior graph constructed in step 2, the edges represent the temporal order and behavior dependency between nodes, and the edge weights reflect the closeness of this relationship. Specifically, for an edge v i →v j in the attack graph, its edge weight w e (v i →v j ) can be used as a reference for the probability of transitioning from the attack stage corresponding to node v i to the attack stage corresponding to node v j . By statistically processing and normalizing the weights of all relevant edges in the attack graph, we can obtain the state transition probability matrix A = [a ij ], where a ij = P(q j |q i ).​

[0152] The state transition probability matrix generation method is as follows:

[0153] (1) Edge weight collection: Extract all directed edges v i →v j from the attack behavior graph, and the weight w e (v i ,v j ) represents the time sequence density and tactical relevance from node v i to v j .

[0154] (2) Stage mapping: Map the graph node v i to the HMM hidden state q i (for example, if the attack type corresponding to the node belongs to "initial access", it is mapped to q1).

[0155] (3) Normalization calculation:

[0156]

[0157] where a ij represents the probability of transitioning from state q i to q j , and the state transition probability matrix A is generated by summing and normalizing the edge weights between the same states.

[0158] 3.2.2 Observation probability P(o t |q t )

[0159] The observation probability P(o t |q t ) represents the probability of observing an alarm cluster feature o t when in attack phase q t . This probability is calculated by the semantic matching degree between the alarm cluster and the attack phase. The specific calculation process is as follows:

[0160] Feature extraction and encoding: Further extract and encode the features of the alarm cluster, and convert them into vector representation suitable for semantic matching. For example, for attack type labels, a word vector model (such as Word2Vec) can be used to convert them into vectors; for timestamp, source IP and other features, normalization and encoding processing can be performed.

[0161] Semantic matching model: Use a pre-trained semantic matching model (such as a neural network model based on deep learning) to calculate the semantic matching degree between the alarm cluster feature vector and each attack phase. This model will learn the common alarm cluster feature patterns under different attack phases, and output the matching scores of the alarm cluster features and each attack phase.

[0162] Probability calculation: Normalize the semantic matching scores to obtain the observation probability P(o t |q t ).

[0163] 3.3 Algorithm implementation and optimization

[0164] 3.3.1 Implementation steps of Viterbi algorithm

[0165] 1. Initialization: At time t = 1, for each hidden state q i , calculate the initial probability δ1(i) = π i ·b i (o1), where π i is the initial state probability, b i (o1) = P(o1|q i ) is the observation probability. At the same time, record the path pointer ψ1(i) = 0.

[0166] 2. Recursion: For t = 2, 3, …, T, for each hidden state q j , calculate and record the path pointer

[0167] 3. Termination: Calculate the final maximum probability and record the final state

[0168] 4. Backtracking: Starting from the final state , backtrack according to the path pointer ψ t to obtain the most likely hidden state sequence

[0169] 3.3.2 Algorithm optimization

[0170] Data preprocessing: Perform preprocessing operations such as deduplication, filtering, and sorting on the alarm cluster data to reduce data noise and improve the efficiency and accuracy of the algorithm.

[0171] Parallel computing: When calculating the recursion step of the Viterbi algorithm, parallel computing can be used to accelerate the calculation process using multi-core processors or GPUs.

[0172] Model updating: As new alarm data continuously appears, regularly update the state transition probabilities and observation probabilities of the HMM to adapt to the changing attack patterns.

[0173] 3.4 Result evaluation and application

[0174] 3.4.1 Result evaluation

[0175] Accuracy evaluation: Use real attack scenario data for verification, compare the consistency of the most likely attack path obtained by the Viterbi algorithm with the actual attack path. Accuracy, recall rate and other indicators can be used to evaluate the accuracy of the algorithm.

[0176] Reliability evaluation: Analyze the stability and reliability of the algorithm under different data sizes and different attack modes. For example, in the case of small data volume or complex attack mode, whether the performance of the algorithm can still meet the requirements.

[0177] 3.4.2 Result application

[0178] Security situation awareness: According to the most likely attack path, potential attack threats are discovered in time to provide decision support for security administrators. For example, when detecting that the attack enters the data exfiltration stage, take measures to prevent data leakage in time.

[0179] Attack tracing: By analyzing the attack path, the source and attack means of the attacker can be traced back, providing a basis for subsequent investigation and prevention.

[0180] Security policy adjustment: According to the probability reasoning results of the attack chain, adjust the network security policy and strengthen the protection of high-risk attack stages. For example, increase detection and prevention measures for the initial access stage to reduce the chances of attackers entering the network. BRIEF DESCRIPTION OF DRAWINGS

[0181] Figure 1 A flowchart of the method described in the present application, which describes a network forensics and attack chain reconstruction method based on threat alerts, through multi-dimensional aggregation of threat alerts, construction of attack behavior graph and probability reasoning attack chain, finally realizing efficient detection and tracing of network attacks. DETAILED DESCRIPTION

[0182] The present application will be further described below in conjunction with specific embodiments:

[0183] EMBODIMENT

[0184] 1 Experimental environment and data characteristics

[0185] Simulation scenario: Construct a heterogeneous network topology containing cloud host, industrial control device and office terminal, simulate the multi-stage penetration process of advanced persistent threat (APT) attack, covering the following typical attack behaviors:

[0186] Initial access: Use phishing email to carry CVE-2024-0123 exploit code (CVSS score 9.8);

[0187] Lateral movement: Intranet expansion using Windows SMB protocol vulnerability (CVE-2023-34478);

[0188] Data exfiltration: Stolen industrial control data is transmitted through encrypted tunnels (443 port disguised as HTTPS traffic).

[0189] Data composition:

[0190]

[0191]

[0192] 2 Core performance index comparison

[0193] 2.1 Alarm aggregation compression rate

[0194] This method: 92.3%.

[0195] Implementation mechanism: Through the dynamic weight aggregation (time decay factor + semantic similarity) of the improved DBSCAN algorithm, the same type of attack alarm is compressed to 7.7% of the original data volume. For example, 5000 port scanning alarms initiated by the same IP are aggregated into 1 alarm cluster containing time range (20:00-20:15), source IP (192.168.1.10), and attack type (port scanning).

[0196] Using the prefix compression feature of the MPT storage structure (refer to section 3.1 of the document), the continuous log shared path nodes of the same device are further reduced in storage redundancy.

[0197] Traditional method: ≤75%.

[0198] Limitations: Based on static time window (such as 1 hour of same type alarm aggregation) and fixed weight (such as equal weight Euclidean distance), it cannot effectively handle cross-platform time deviation and dynamic attack characteristics, resulting in a high rate of repeated alarm misaggregation of more than 25%.

[0199] Unique verification of improved DBSCAN:

[0200] In scenarios involving APT and ransomware hybrid attacks, traditional DBSCAN uses a fixed alpha = 0.05 / minute, resulting in:

[0201] Cross-day alarms of APT attacks (such as 12-hour intervals) cannot be aggregated due to excessive decay, with a missing aggregation rate of 34%;

[0202] Short-term multi-wave attacks of ransomware (interval <10 minutes) are misaggregated as a single cluster due to insufficient decay, with a misaggregation rate of 18%.

[0203] And this solution through the dynamic alpha value adaptation, APT alarm leakage rate to 4%, ransomware false aggregation rate to 1.1%, to prove the irreplaceability of dynamic attenuation mechanism.

[0204] As shown in the following table:

[0205]

[0206] 2.2 Attack chain reconstruction accuracy

[0207] This method: 88.6%.

[0208] Technical support:

[0209] The timing Petri net is combined with the hidden Markov model (see section 2.1 of the reference document), which accurately models the timing constraints (such as the delay of the jump machine connection needs to be less than 5 minutes) and state transition probabilities (such as the probability of "data exfiltration" after "privilege escalation" is 0.85) of the attack path.

[0210] The Byzantine fault-tolerant verification protocol (see section 2.2 of the reference document) filters 30% of the fake alarms, ensures the integrity of the data chain, and the false positive rate is only 1.4%.

[0211] Baseline method based on rules: 64.2%.

[0212] Limitations: Relies on manually defined attack rules (such as "port scanning -> exploit -> privilege escalation"), cannot identify variant attack paths (such as directly entering the "persistence" stage through supply chain vulnerabilities), and the false negative rate is as high as 35.8%.

[0213] As shown in the following table:

[0214]

[0215] 2.3 Time consumption of single analysis

[0216] This method: <3 seconds.

[0217] Optimization measures:

[0218] Pipelining architecture design (see section 3.4 of the reference document) realizes the parallel processing of alarm aggregation, graph construction, and probability reasoning, and the time consumption of each stage is: aggregation 1.2 seconds, graph construction 0.9 seconds, and reasoning 0.8 seconds.

[0219] Lightweight algorithm design: The time complexity of the improved DBSCAN is O(n log n), and the matrix fast power optimization is used in the Viterbi algorithm, which is 40% faster than the traditional implementation.

[0220] Open source tool Maltrail: 15 seconds.

[0221] Performance bottleneck: Based on single-thread rule matching, it takes 3 layers of regular expressions to traverse 500,000 data, and lacks dynamic weight aggregation mechanism, resulting in long time consumption.

[0222] 2.4 Key algorithm parameter settings

[0223]

[0224]

[0225] 3 Typical attack scenario verification results

[0226] Scenario: Simulate APT attack chain containing 7 attack stages (initial access → execution → persistence → lateral movement → privilege escalation → data collection → exfiltration), inject 200,000 fake alerts (40%).

[0227]

[0228] 4 Performance advantage summary

[0229]

[0230]

[0231] Conclusion: Through the above design, the improved DBSCAN of this scheme is not a simple combination of existing technologies, but a deep optimization for the "time sensitivity difference" and "cross-stage semantic association" of network attacks. Specifically:

[0232] Dynamic alpha value adjustment logic: Through attack type labels (such as CVE / CWE), automatically match decay parameters alpha (APT attack alpha = 0.01 / minute, ransomware alpha = 0.1 / minute), solve the problem that traditional fixed decay rate cannot distinguish attack type time sensitive characteristics. In mixed attack scenarios, this mechanism reduces APT alert missing aggregation rate from 34% to 4%, ransomware false aggregation rate from 18% to 1.1%, significantly improving the accuracy of cross-period aggregation.

[0233] LSTM semantic learning ability: Through a 2-layer bidirectional LSTM network (128 neurons per layer), learn the time-dependent sequence of historical attack sequences, such as automatically increasing the source IP weight ω_s from 0.2 to 0.5 in cross-platform attacks, increasing the source IP association accuracy from 63% to 89%. Compared with traditional static weight method, this mechanism realizes dynamic modeling of attack behavior semantic features (such as cross-system penetration path).

[0234] Combined with the comparative experimental data (the compression ratio of the improved DBSCAN is 92.3%, and the error aggregation rate is 1.2%, which are significantly better than K-means and the original DBSCAN), the technical improvement of the scheme reflects the innovative response to the essential characteristics of network attacks, and can effectively support the demonstration of non-obviousness.

[0235] Embodiments of the application are not limited to the above description, and the values of the dynamic weight coefficients (ω t , ω s , ω d , ω l ) and the time decay parameter α in the improved DBSCAN algorithm, the setting of the node weight calculation coefficient α in the attack behavior graph construction, the number of neurons, the Dropout rate of the LSTM network in the joint modeling of the time sequence Petri net and the HMM, and the update frequency of the state transition probability matrix in the Viterbi algorithm can be adjusted according to the actual network security scene. Such improvements are within the protection scope of the application.

Claims

1. A threat alert based network forensics and attack chain reconstruction method, characterized in that, The method comprises the following steps: The threat alert multi-dimensional aggregation includes the following steps: The attack behavior graph construction includes the following steps: The attack chain probability reasoning includes the following steps:

2. The method of claim 1, wherein, In the threat alert multi-dimensional aggregation, the attributes of the original threat alert include a timestamp, a source IP, a destination port, an attack type label and additional metadata, and the preprocessing includes unstructured field regularized analysis and Bloom filter deduplication.

3. The method of claim 1, wherein, The dynamic weight adjustment mechanism of the improved DBSCAN algorithm is realized by entropy weight method initialization and LSTM dynamic optimization, and the weight coefficient satisfies ω t +ω s +ω d +ω l =1; the time decay factor combines the dynamic adjustment of the decay parameter α, and the APT attack α=0.01 / min, and the ransomware α=0.1 / min.

4. The method of claim 1, wherein, In the attack behavior graph construction, the node mapping rule is to map each alert cluster to a graph node, and the node includes the core features and additional information of the alert cluster; and the edge construction rule is to construct a directed edge if the time of the previous alert cluster is earlier than that of the subsequent alert cluster and the attack types have a behavior association.

5. The method of claim 1, wherein, The node weight is calculated by the CVSS score and frequency of the alarm cluster, and the formula is The edge weight is calculated based on the time sequence density and the tactical stage relevance, and the formula is 6. The method of claim 1, wherein, The time sequence Petri net is used for modeling the time sequence constraint of the attack stage, and the HMM learns the state transition probability through the edge weight of the attack behavior graph; the hidden state represents different stages of the attack process, including initial access, execution, persistence, lateral movement, privilege escalation and data exfiltration; and the observation state is composed of the features of the alert cluster.

7. The method of claim 1, wherein, The LSTM network is a 2-layer bidirectional LSTM layer + 1-layer fully connected layer, the input is an alarm feature vector, the output is a 4-dimensional weight adjustment coefficient Δω = (Δω t , Δω s , Δω d , Δω l ), the training data is the real attack log of the past 12 months, and the loss function includes the mean square error and the L1 regularization term.

8. The method of claim 1, wherein, The Viterbi algorithm solves the most probable attack path through the initialization, recursion, termination and backtracking steps, the state transition probability is obtained by statistical normalization of the edge weight of the attack behavior graph, and the observation probability is calculated according to the semantic matching degree of the alert cluster and the attack stage.

9. A computer-readable storage medium, characterized in that, The medium stores a computer program configured to execute the steps of the threat alert-based network forensics and attack chain reconstruction method according to any one of claims 1-8.

10. A computer program product, characterised in that, The product includes computer executable instructions for realizing the functions of the threat alert-based network forensics and attack chain reconstruction method according to any one of claims 1-8.

Images