A dynamic deception enhanced network attack unsupervised clustering and tracing method and system

Abstract

This invention discloses an unsupervised clustering and attribution method and system for dynamic deception-enhanced network attacks. The invention identifies suspicious behavioral clusters by real-time collection of raw network logs and unsupervised incremental clustering. When the suspiciousness exceeds a threshold, deception resources matching the behavioral pattern are automatically and dynamically deployed to capture deep interaction sequences of the attacker, generating enhanced data. The features of the raw logs and the enhanced data are spatiotemporally aligned and fused to generate an intent feature vector representing the attacker's tactics, techniques, and processes. Based on this vector, refined clustering and attacker attribution are performed. This invention solves the problems of inaccurate clustering intent identification and low attribution accuracy caused by the lack of deep interaction data in existing technologies, significantly improving the ability to discover unknown threats and the accuracy of attack attribution.

Description

Technical Field

[0001] This invention relates to the field of computer network security technology, specifically to an unsupervised clustering and tracing method and system for dynamic deception-enhanced network attacks. Background Technology

[0002] Currently, cybersecurity threats, especially advanced persistent attacks (APTs), zero-day exploits, and malicious behaviors with unknown patterns, have become major challenges for network protection. To address the increasingly complex attack landscape, the industry has developed various detection and attribution technologies, mainly including the following categories:

[0003] (I) Detection techniques based on signatures and rules

[0004] Traditional security measures, such as Intrusion Detection Systems (IDS), Intrusion Prevention Systems (IPS), and Security Information and Event Management (SIEM) systems, primarily rely on known attack signature databases or manually defined rules. These methods are highly efficient at detecting attacks with existing patterns, but they cannot identify unknown or variant attacks. Their detection capabilities depend on the timeliness of rule database updates, exhibiting a significant lag. Furthermore, these systems primarily rely on passive log analysis, making it difficult to proactively lure or deeply investigate attacker behavior.

[0005] (II) Anomaly Detection Techniques Based on Unsupervised Learning

[0006] To address the blind spot of signature detection in the face of unknown attacks, researchers have introduced unsupervised learning methods, such as density-based clustering (DBSCAN) and the CluStream algorithm based on streaming data. These methods do not require labeled data and can detect anomalous behaviors deviating from the normal baseline. However, existing unsupervised clustering techniques face a "data hunger" dilemma in cybersecurity applications: their clustering effectiveness heavily depends on the quality and richness of input features. Conventional network logs (such as NetFlow, firewall logs, and web proxy logs) only contain metadata-level statistical information (such as IP addresses, ports, and packet counts), lacking deep interaction semantics between attackers and target systems (such as specific commands, system calls, file access paths, and vulnerability detection behaviors). This lack of contextual information makes it difficult for clustering models to accurately distinguish between different attack intentions such as scanning, penetration, and lateral movement, and also causes attack attribution to often remain at the IP address level, failing to effectively link multiple activities of the same attacker.

[0007] (III) Active defense based on honeypot and deception technology

[0008] Honeypots, honeynets, and other deception techniques lure attackers into revealing their behavior by deploying decoy resources, thereby collecting high-fidelity attack data. However, current deception systems generally operate statically and pre-deployed, independent of real-time detection systems. Their shortcomings include: the inability to dynamically adjust deception resources based on real-time anomalies or new attack patterns; the failure to automatically feed the generated interactive data back to the detection model for online learning or updates; and a lack of intelligent deployment strategies, resulting in inefficient collection of high-value attack data and hindering the formation of a closed-loop defense system that integrates detection and deception.

[0009] In summary, the existing technology has the following main technical bottlenecks:

[0010] Passivity and lag: Traditional detection methods are slow to respond to unknown threats and cannot proactively uncover potential attack behaviors.

[0011] Unsupervised clustering is difficult to understand: due to the lack of deep interactive information in the input data, the clustering results are difficult to reflect the attacker's real tactics, techniques and processes (TTPs), resulting in low source tracing accuracy.

[0012] Deception techniques are isolated and static: proactive defense methods such as honeypots cannot be dynamically linked with detection models, failing to form a closed-loop mechanism of "detection triggering deception, deception enhancing data, and data optimizing detection".

[0013] The methods of tracing the source are limited: most source tracing is based only on crude features such as IP address and timestamp, and cannot utilize high-fidelity interactive behavior to construct the attacker's behavioral DNA. Summary of the Invention

[0014] To address this, embodiments of the present invention provide an unsupervised clustering and source tracing method and system for dynamic deception-enhanced network attacks, thereby solving the technical problems of inaccurate clustering intent identification and low source tracing accuracy in existing technologies.

[0015] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0016] According to a first aspect of the present invention, an unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks is provided, the method comprising:

[0017] Real-time collection of raw network logs and traffic data, followed by streaming processing and standardization;

[0018] Incremental clustering algorithm is used to perform real-time clustering on the processed data, identify suspicious behavior microclusters that deviate from the normal baseline, and calculate the suspicion score of each microcluster.

[0019] When the suspiciousness score of a micro-cluster exceeds a preset threshold, a deception scheme matching the suspicious behavior pattern is automatically generated, and deception resources are dynamically instantiated on a software-defined network or cloud infrastructure.

[0020] Record the attacker's deep interaction sequence within the deceptive resource, including command history, system calls, file operations, and internal network connection attempts, to form high-fidelity enhanced data;

[0021] The original log features are spatiotemporally aligned and fused with the enhanced data to generate an intent feature vector that characterizes the attacker's tactics, techniques and processes. Based on this intent feature vector, the suspicious micro-clusters are refined and clustered to obtain the attacker intent cluster.

[0022] Within the intent cluster, an attack event attribute relationship graph is constructed through multi-level association analysis, and a community discovery algorithm is applied to identify the set of entities belonging to the same attacker or attack organization, generating an attacker behavior fingerprint.

[0023] Furthermore, the construction of the original log features includes: extracting traffic scale features, target dispersion features, protocol and flag features, and time-series behavior features for each monitored entity within a time window; the target dispersion features include the number of unique target IPs, the number of unique target ports, the target port entropy, and the target IP entropy; the time-series behavior features include activity burstiness, activity period autocorrelation coefficient, connection interval statistical features, and port access sequence patterns.

[0024] Furthermore, the feature extraction of the enhanced interactive data includes: performing word embedding encoding and n-gram pattern extraction on the attacker's command history; constructing a directed attribute graph from the system call sequence and file operation records, and extracting the number of nodes, number of edges, average degree, diameter and clustering coefficient of the graph; mapping the interactive behavior to the specific techniques of the MITRE ATT&CK framework, and generating binary feature vectors.

[0025] Furthermore, the feature fusion is achieved using a self-supervised encoder: the original log features and the augmented data features are respectively input into a multilayer perceptron for dimensionality reduction, concatenated and further fused to output a low-dimensional intent representation vector; the encoder is jointly trained through augmented data reconstruction tasks and contrastive learning tasks, which reduces the spatial distance between the intent representation vectors of the same attacker in different time windows and increases the spatial distance between the intent representation vectors of different attackers.

[0026] Furthermore, the incremental clustering algorithm employs a micro-cluster maintenance mechanism: each micro-cluster maintains the linear sum and square sum of its feature vectors, the number of data points, and the last update timestamp; when a new data point arrives, its distance to the center of an existing micro-cluster is calculated; if the distance is less than the product of the micro-cluster radius and the expansion factor, the point is absorbed; otherwise, a new micro-cluster is created; micro-clusters that are too close to each other are merged periodically, and micro-clusters that have not been updated for a long time are archived and cleaned up.

[0027] Furthermore, the refined clustering adopts an intent-aware composite distance metric, which integrates basic feature distance, temporal correlation distance, and intent similarity based on TTPs; wherein the intent similarity is calculated based on whether the attacker performs the same or similar ATT&CK techniques, and the weight of intent similarity dynamically increases when the augmented data is rich.

[0028] Furthermore, the multi-level correlation analysis includes: correlation at the IP and infrastructure level, including the same network segment, shared certificates, or fingerprint information; correlation at the behavioral pattern level, including attack time patterns, tool usage fingerprints, and error message characteristics; and correlation at the tactical, technical, and process level, including the matching degree of ATT&CK tactical phases and the similarity of vulnerability exploitation methods.

[0029] Furthermore, the method also includes: real-time calculation of the silhouette coefficient, Davidson-Bolding index, and intention index of the clustering results. Figure 1 Consistency index, when Figure 1 When the consistency index decreases, the weight of intent similarity in refined clustering is automatically increased. Figure 1 When the similarity index is good, the weight should be appropriately reduced, and the bandwidth parameter for similarity calculation should be adjusted using a local scale adaptive method.

[0030] According to a second aspect of the present invention, an unsupervised clustering and attribution system for dynamic deception-enhanced network attacks is provided, the system comprising:

[0031] The data acquisition and computing module is used to collect raw logs and traffic from network devices, hosts and cloud platforms, and perform real-time streaming processing and standardization.

[0032] The intelligent analysis and decision-making module includes an unsupervised clustering analysis engine and a dynamic deception decision engine. The unsupervised clustering analysis engine is used to perform real-time clustering on the processed data and identify suspicious behavior micro-clusters. The dynamic deception decision engine is used to generate deception schemes based on the suspiciousness scores of the micro-clusters.

[0033] The proactive response and execution module includes a deception environment orchestrator and an enhanced data collector. The deception environment orchestrator is used to automatically deploy deception resources according to the deception scheme, and the enhanced data collector is used to record the attacker's deep interaction behavior in the deception environment and output enhanced data.

[0034] The feedback loop module transmits the enhanced data back to the intelligent analysis and decision-making module to optimize the feature representation and clustering model of the unsupervised clustering analysis engine.

[0035] Furthermore, the intelligent analysis and decision-making module also includes an agent network module, which takes the original log features as input and is trained to approximate the intent representation vector output by the feature fusion encoder, thereby generating approximate intent features for network entities that have not yet triggered deception resources, for preliminary clustering and suspiciousness assessment.

[0036] The embodiments of the present invention have the following advantages:

[0037] 1) From passive to active: Transform active deception technology from a static, isolated tool into a dynamic, intelligent component integrated into the detection closed loop.

[0038] 2) Enhance the ability to detect unknown threats: By discovering suspicious clues through unsupervised clustering and using dynamic deception for "verification" and "in-depth investigation", it is possible to effectively discover unknown attacks that are missed by traditional methods.

[0039] 3) Significantly enhances the accuracy of source tracing: By utilizing high-fidelity TTPs data obtained in the deceptive environment, it provides attacker profiling with a "behavioral DNA" that is far more reliable than IP addresses, making it more accurate to link different attack events of the same attacker.

[0040] 4) Automation and Efficiency: It has achieved full-process automation from threat detection and automatic response (deployment deception) to data collection and model optimization, which greatly improves Security Operations Efficiency (SOAR).

[0041] 5) A continuously evolving defense system: It has formed a self-reinforcing closed loop of "data-driven model, model-guided deception, and deception generating new data", which enables the entire defense system to continuously learn and adapt to new types of attacks. Attached Figure Description

[0042] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0043] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0044] Figure 1 This is a schematic diagram of the logical structure of an unsupervised clustering and tracing system for dynamic deception-enhanced network attacks provided in an embodiment of the present invention;

[0045] Figure 2 This is a deployment diagram of an unsupervised clustering and tracing system for dynamic deception-enhanced network attacks provided by an embodiment of the present invention;

[0046] Figure 3 This is a flowchart illustrating an unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks provided in an embodiment of the present invention. Detailed Implementation

[0047] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] refer to Figure 1 and Figure 2 This invention discloses an unsupervised clustering and tracing system for dynamic deception-enhanced network attacks. The system includes: a data acquisition and calculation module 1; an intelligent analysis and decision-making module 2; an active response and execution module 3; and a feedback closed-loop module 4.

[0049] Corresponding to the aforementioned unsupervised clustering and tracing system for dynamic deception-enhanced network attacks, this invention also discloses an unsupervised clustering and tracing method for dynamic deception-enhanced network attacks. The following details the unsupervised clustering and tracing method for dynamic deception-enhanced network attacks disclosed in this invention, in conjunction with the aforementioned unsupervised clustering and tracing system for dynamic deception-enhanced network attacks.

[0050] refer to Figure 3 This invention discloses an unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks, which collects raw network logs and traffic data in real time and performs streaming processing and standardization.

[0051] Incremental clustering algorithm is used to perform real-time clustering on the processed data, identify suspicious behavior microclusters that deviate from the normal baseline, and calculate the suspicion score of each microcluster.

[0052] When the suspiciousness score of a micro-cluster exceeds a preset threshold, a deception scheme matching the suspicious behavior pattern is automatically generated, and deception resources are dynamically instantiated on a software-defined network or cloud infrastructure.

[0053] Record the attacker's deep interaction sequence within the deceptive resource, including command history, system calls, file operations, and internal network connection attempts, to form high-fidelity enhanced data;

[0054] The original log features are spatiotemporally aligned and fused with the enhanced data to generate an intent feature vector that characterizes the attacker's tactics, techniques and processes. Based on this intent feature vector, the suspicious micro-clusters are refined and clustered to obtain the attacker intent cluster.

[0055] Within the intent cluster, an attack event attribute relationship graph is constructed through multi-level association analysis, and a community discovery algorithm is applied to identify the set of entities belonging to the same attacker or attack organization, generating an attacker behavior fingerprint.

[0056] This invention is logically divided into three layers: a data acquisition and computation layer, an intelligent analysis and decision-making layer, and a proactive response and execution layer. These layers communicate through a unified message bus and API, forming a loosely coupled, scalable closed-loop logic.

[0057] Data Acquisition and Computation Layer: Responsible for collecting raw logs and traffic from network devices, hosts, and cloud platforms, and performing real-time streaming processing and standardization.

[0058] Intelligent Analysis and Decision Layer: The core consists of an unsupervised clustering analysis engine and a dynamic deception decision engine. The former processes data in real time and outputs suspicious clusters; the latter determines whether to trigger deception and the deception scheme based on the strategy.

[0059] Active Response and Execution Layer: Composed of a deception environment orchestrator and an enhanced data collector, it is responsible for instantiating, configuring, and managing deception resources on software-defined networks and cloud infrastructure, and collecting high-fidelity interactive data.

[0060] Detailed implementation plan for the unsupervised clustering module:

[0061] Objective: To perform real-time clustering of massive, multi-dimensional, time-series network entity (such as IP pairs and sessions) behaviors to discover groups that deviate from the baseline or have suspicious collaborative patterns.

[0062] Feature engineering:

[0063] The feature engineering of this invention is divided into three levels, which process data of different granularities and sources respectively, and finally merge them into a unified behavior representation vector:

[0064] L1 - Raw Log Layer Features: Statistical and time-series features extracted from raw data such as network traffic and security device logs.

[0065] L2-Enhanced Interaction Layer Features: Structured and semantic features extracted from deep interaction behaviors of attackers captured in dynamic deception environments.

[0066] L3-Fusion and Intent Layer Features: Align and fuse L1 and L2 features, and extract high-order intent representations through a deep learning model.

[0067] L1-raw log layer feature construction:

[0068] Input: Raw network log streams (such as NetFlow, firewall logs, proxy logs) in the format of a time-series event sequence: ;

[0069] Output: Each monitored entity (from an external source) (For example) the feature vector within the time window W .

[0070] Processing procedure:

[0071] Time window segmentation and event aggregation:

[0072] Set a fixed sliding window W (e.g., 5 minutes). For each window, specify the source IP address. Aggregate all its outbound events:

[0073] Basic statistical characteristics (12 dimensions in total)

[0074] 1. Traffic Scale Characteristics:

[0075] f1:

[0076] f2:

[0077] f3:

[0078] f4:

[0079] 2. Target dispersion characteristics (reflecting scanning or detection behavior):

[0080] f5:

[0081] f6:

[0082] f7: ,in It is the frequency of accessing port k.

[0083] f8: ,in Is it a visit The frequency.

[0084] 3. Protocol and Marking Characteristics:

[0085] f9: The ratio of the SYN flag in a TCP connection to the total number of connections (reflecting connection establishment attempts).

[0086] f10: The percentage of RST flags in a TCP connection (reflecting abnormal termination).

[0087] f11: ICMP request packet ratio.

[0088] f12: Connection ratio for uncommon protocols (not TCP / 80, 443, 22, 53, etc.).

[0089] Temporal behavioral characteristics (8 dimensions in total)

[0090] Activity Intensity Mode:

[0091] Subdivide window W into M sub-windows (e.g., 30 seconds each). Calculate the number of connections in each sub-window to obtain a time series of length M. .

[0092] f13:

[0093] f14: .

[0094] Connection interval characteristics:

[0095] Calculate the time interval sequence between consecutive events .

[0096] f15:

[0097] f16:

[0098] f17: (Reflects the symmetry of the interval distribution and is often used to detect low-speed scans).

[0099] Port access sequence characteristics:

[0100] Sort the events by time and extract the target port sequence. .

[0101] f18: (A positive value indicates that port scanning may proceed from low to high).

[0102] f19: Sequence self-similarity = ... Divide the sample into two halves and calculate the Jaccard similarity.

[0103] f20: Common attack port pattern matching degree = Calculation Edit distance from predefined scan patterns (such as [22, 80,443, 8080]).

[0104] final, It has 20 dimensions in total. Each feature needs to be Z-score standardized to eliminate dimensions.

[0105] L2-Enhanced Interaction Layer Feature Construction

[0106] Input: Captured from a dynamic deception environment, related to a specific source Related deep interaction data ,Include:

[0107] Command history:

[0108] System call sequence:

[0109] File access history:

[0110] Network connection attempt (internal network):

[0111] Output: Feature vectors representing the attacker's tactics, techniques, and processes (TTPs) .

[0112] Processing procedure:

[0113] Command line as features (10+ dimensions in total)

[0114] Basic statistical characteristics:

[0115] g1: Total number of commands m

[0116] g2: Unique command number

[0117] g3: Average command length (number of characters)

[0118] g4-g8: The count or proportion of each category of commands.

[0119] Command embedding vectors: Using a pre-trained command-line word embedding model (trained on a secure corpus), each command is embedded... Convert to vector For the entire sequence The embedding vector is averaged or max-pooled to obtain g9-gk (k-dimensional, such as 128-dimensional).

[0120] Command sequence pattern characteristics:

[0121] Will Treat the text as text, extract 2-gram and 3-gram features, and retain the top-N most frequent n-grams as boolean features (1 if they appear, 0 otherwise). For example, "curl && wget", "whoami && ls".

[0122] System call graph features (6+ dimensions in total)

[0123] Graph construction: and Merge and construct a directed property graph .

[0124] Node V: Process, file, network socket.

[0125] Edge E: System calls (such as execve, open, connect).

[0126] Statistical characteristics of the graph:

[0127] g11: Total number of graph nodes

[0128] g12: Total number of edges in the graph

[0129] g13: Average

[0130] g14: Diameter of the graph (longest and shortest paths)

[0131] g15: Clustering coefficient

[0132] g16: In-degree and out-degree statistics for sensitive file nodes (such as / etc / shadow, / root / .bash_history).

[0133] MITRE ATT&CK TTPs mapping features (approximately 20 dimensions in total)

[0134] TTPs Matching: Establish a rules engine to map interactive behaviors to specific technologies (T numbers) in the MITRE ATT&CK framework.

[0135] For example: the command nmap -sS 192.168.1.0 / 24 -> T1046 (Network Service Scanning)

[0136] Command cat / etc / passwd-> T1003.008 (OS Credential Dumping: / etc / passwd)

[0137] Try connecting to the internal SMB port 445 -> T1021.002 (SMB / Windows Admin Shares)

[0138] Feature generation: Create a binary feature for each relevant ATT&CK technique (selecting Top-K common techniques), assigning a value of 1 if the technique appears in the session, and 0 otherwise. g17-g36.

[0139] final, It is a concatenation of all the above features, and the dimension d_enh may reach 100-200 dimensions. It also needs to be standardized.

[0140] L3-Fusion and Intent Layer Feature Construction

[0141] Objective: To and Align them in time and space and merge them into a unified feature vector that can comprehensively represent "who (source IP) did what (deep interaction intent) when (window W) and in what way (original traffic pattern)".

[0142] Spatiotemporal alignment algorithm

[0143] Input: Raw log entity Features of window W and a series of enhanced data sessions S= Each session Associate a source IP . and time interval .

[0144] algorithm:

[0145] # For each raw log entity (s_ip, W):

[0146] # 1. Find matching enhanced sessions:

[0147] Matching_Sessions = []

[0148] for sess_j in S:

[0149] if sess_j.attacker_ip == s_ip AND

[0150] (sess_j.start ∈ W OR sess_j.end ∈ W OR W is contained in [sess_j.start, sess_j.end]):

[0151] Matching_Sessions.append(sess_j)

[0152] # 2. If Matching_Sessions is not empty:

[0153] # For each matching session sess_j, extract its V_enhanced(sess_j) (i.e. V_enhanced(s_ip)).

[0154] # Calculate the enhanced features V_enh_agg after aggregation:

[0155] # - If there is only one session: V_enh_agg = V_enhanced(sess_j)

[0156] # - If there are multiple sessions: you can take the average or a weighted average based on session duration.

[0157] # Forming a fusion pair: P = (s_ip, W, V_raw(s_ip, W), V_enh_agg, label=1) # label=1 indicates that there is augmented data

[0158] # 3. Otherwise:

[0159] # Form partial data pairs: P = (s_ip, W, V_raw(s_ip, W), None, label=0) # label=0 indicates no augmented data

[0160] Feature fusion and dimensionality reduction

[0161] For the fusion pair P with label=1, we need to... and Merging. Simple splicing can lead to the curse of dimensionality and the "curse of dimensionality."

[0162] Solution: Use a multilayer perceptron (MLP) encoder for feature fusion and compression.

[0163] Input layer:

[0164] Input after standardization.

[0165] Input after standardization.

[0166] Encoder structure:

[0167] h_raw = ReLU(W1_raw * V_raw + b1_raw) # Fully connected layer, dimensionality reduction

[0168] h_enh = ReLU(W1_enh * V_enh_agg + b1_enh) # Fully connected layer, dimensionality reduction

[0169] h_concat = Concat(h_raw, h_enh) # Concatenate along the feature dimensions

[0170] h_fused = ReLU(W2 * h_concat + b2) # Further fusion

[0171] z_intent = ReLU(W3 * h_fused + b3) # Output a low-dimensional intent representation vector, such as 32-dimensional.

[0172] Here, z_intent is the high-order, dense, intent-aware feature vector we ultimately use for clustering. It integrates network behavior statistics and attack interaction semantics.

[0173] Training the encoder: Since we do not have We train this encoder using self-supervised learning to obtain the real labels.

[0174] Self-supervised task 1: Augmented data reconstruction. For data with label=1, design a decoder to attempt to reconstruct data from... Refactoring The loss function is the mean squared error (MSE).

[0175] Self-supervised task 2: Contrastive learning. Construct positive and negative sample pairs. Positive sample pairs: samples taken by the same attacker in different windows. and Negative sample pairs: data from different attackers. The encoder is trained to make the z_intent vectors of positive samples closer together in space and the negative samples farther apart. The loss function is the InfoNCE loss.

[0176] Through joint training on these two tasks, the encoder learns to generate a feature representation that both preserves the original statistical information and highly summarizes the attacker's intent. .

[0177] The features ultimately used for clustering:

[0178] For an entity with label=1 (with augmented data), the final feature is: .

[0179] For entities with label=0 (no augmented data), we cannot compute... When the initial model is not yet properly trained, it can be used temporarily. Once the encoder is trained to a mature state, we can use a proxy network that only uses... The input is used, but it is trained to approximate the encoder E (the input is...). ) output That is, minimize In this way, even for new entities that have not yet triggered deception, we can use... This generates a feature that approximates intent perception, which is then used for initial clustering.

[0180] Fast Incremental Clustering Module:

[0181] Objective: To perform rapid preliminary clustering of feature vectors arriving in real time, identify abnormal behavior patterns, and trigger dynamic deception with low latency.

[0182] Algorithm Fundamentals: Incremental Micro-cluster Maintenance

[0183] The improved CluStream algorithm is used to maintain two types of data structures:

[0184] microclusters Statistical summary:

[0185] For each microcluster Maintain a tuple :

[0186] Linear sum of eigenvectors;

[0187] Sum of squares of eigenvectors;

[0188] Number of data points;

[0189] t: Last update timestamp.

[0190] Pyramid Time Window:

[0191] To handle different time granularities, maintain micro-cluster summaries for multiple time windows:

[0192] Last 5 minutes: High-precision window

[0193] Last hour: Medium precision window

[0194] Recent 24 hours: Low-precision window

[0195] Historical archiving: Long-term mode storage

[0196] Online clustering process:

[0197] Input: The feature vector arriving at time t .

[0198] Step 1: Find the nearest cluster

[0199] Calculate z_t to all existing microclusters Distance from the center:

[0200]

[0201] in Find the k nearest microclusters (k=3).

[0202] Step 2: Determining the affiliation of microclusters

[0203] For each candidate microcluster Calculate its radius:

[0204]

[0205] in It can be calculated from CF2, if ,but Can be absorb.

[0206] Step 3: Micro-cluster update or creation

[0207] Scenario A: Can be absorbed by existing microclusters

[0208] Update the latest microclusters :

[0209]

[0210]

[0211]

[0212]

[0213] Case B: Cannot be absorbed by any microcluster

[0214] Create new microclusters :

[0215]

[0216]

[0217]

[0218]

[0219] Step 4: Cluster Merging and Cleaning

[0220] Periodic merging: Every T_merge time (e.g., 10 minutes), merge microclusters that are too close together.

[0221] For all microcluster pairs :

[0222] if :

[0223] merge and :

[0224]

[0225]

[0226]

[0227] Aging cleanup: Microclusters that have not been updated for a long time (e.g., 24 hours) are archived to the historical database and removed from the active set.

[0228] Step 5: Anomaly Detection and Discovery of New Attack Patterns

[0229] Outlier detection: after creation Not enough points were absorbed within the time limit (e.g.) Microclusters of ) are marked as candidate anomalous microclusters.

[0230] Microcluster Suspicion Score:

[0231]

[0232] in:

[0233]

[0234]

[0235]

[0236] when At that time, the dynamic deception module is triggered, and... The feature summary is passed to the intent-aware refined clustering module.

[0237] Intent-aware refined clustering module:

[0238] Objective: To refine and cluster attack micro-clusters discovered in the first layer using enhanced data obtained through dynamic deception, thereby enabling intent-based attacker correlation.

[0239] Similarity metrics for enhanced intent:

[0240] Core innovation: Design a composite distance metric that integrates behavioral statistical similarity and attack intent similarity.

[0241] Definition 1: Basic Feature Distance

[0242] For two data points and (from) or ):

[0243]

[0244] in The feature weights are calculated using the information gain ratio.

[0245] Definition 2: Temporal correlation distance

[0246] Consider the temporal correlation of the attack behavior:

[0247]

[0248] Where λ is the attenuation coefficient, controlling the degree of influence of time.

[0249] Definition 3: Intent Similarity (Key Innovation)

[0250] For point pairs with augmented data, calculate the intent distance based on TTPs:

[0251] Let the TTPs eigenvector of point i be... ,in Indicates whether the m-th type of TTP occurs.

[0252] Let the TTPs eigenvector of point j be...

[0253]

[0254] For command sequence similarity (when TTPs features are insufficient):

[0255] Let the command sequence for point i be...

[0256] Let the command sequence for point j be...

[0257] Using improved edit distance:

[0258]

[0259] Where Lev is the weighted edit distance that takes into account command semantics.

[0260] Definition 4: Composite Distance Formula

[0261] Ultimate intention perception distance:

[0262]

[0263] in ,and The weights are dynamically adjusted according to the richness of the augmented data:

[0264]

[0265] Number of augmented data points collected so far.

[0266] Adjustment coefficient

[0267] Refined clustering algorithm based on spectral clustering:

[0268] Since attacker groups may form complex nonlinear structures, spectral clustering is used for refinement.

[0269] Step 1: Construct a similarity matrix

[0270] For the suspicious micro-cluster set output from the first layer Calculate representative points (such as the center of a micro-cluster) and construct a similarity matrix W:

[0271]

[0272] Where σ is the scale parameter and D is the composite distance mentioned above.

[0273] Step 2: Enhance data dissemination

[0274] For points with augmented data, their similarity information is propagated to similar points without augmented data:

[0275] Let E be the point set with augmented data.

[0276] For each point Correct its similarity to other points:

[0277]

[0278] This allows the intent information to spread throughout the graph.

[0279] Step 3: Solve using spectral clustering

[0280] Calculate the degree matrix D, where ,

[0281] Calculate the normalized Laplace matrix: ,

[0282] Calculate the eigenvectors corresponding to the first K smallest eigenvalues ​​of L. ,

[0283] Form a matrix from the eigenvectors ,

[0284] K-means clustering is performed on the row vectors of U to obtain the final refined clustering result.

[0285] Step 4: Automatically determine the number of clusters K

[0286] The optimal number of clusters is automatically determined using the characteristic gap method.

[0287] set up Eigenvalues ​​of L

[0288] Calculate the eigenvalue gap:

[0289] Choose i that maximizes gap(i) as K.

[0290] Simultaneously satisfy: (To prevent too many small clusters)

[0291] Incremental spectral clustering optimization:

[0292] Traditional spectral clustering has high computational complexity, making it unsuitable for online scenarios. This invention proposes an incremental optimization scheme:

[0293] When a new data point z_{new} is added, the entire eigenvalue decomposition is not recalculated:

[0294] calculate Similarity vector with existing anchor points

[0295] renew

[0296] The eigenvectors are updated approximately using matrix perturbation theory:

[0297] Let the original feature pairs be

[0298] The disturbance caused by the new point is ΔW

[0299] Then the new feature vector

[0300] Recalculate the exact solution periodically (e.g., every 1000 new points).

[0301] Attacker attribution correlation algorithm:

[0302] Objective: To further analyze the relationships between attacker entities within the same refined cluster.

[0303] Multi-level association analysis:

[0304] Level 1: IP and Infrastructure Connection

[0305] IP address is the same as the C / AS number.

[0306] Share SSL certificates and Whois information.

[0307] Same User-Agent, HTTP header fingerprint.

[0308] Level 2: Behavioral Pattern Association

[0309] Attack time patterns (time zone, working hours).

[0310] Tool usage fingerprint (command sequence, parameter preferences).

[0311] Error message characteristics.

[0312] Level 3: TTPs Tactical Connections

[0313] ATT&CK tactical phase compatibility

[0314] Similarity of vulnerability exploitation methods

[0315] C2 communication mode matching.

[0316] Connection graph construction and analysis:

[0317] Step 1: Construct the attribute relationship graph

[0318]

[0319] V: Attack event node;

[0320] E: Edge, with weights representing the similarity between events;

[0321] A: Node attributes (IP, time, TTPs, etc.).

[0322] Step 2: Community Detection Algorithm

[0323] Apply the Louvain community detection algorithm to the association graph to identify closely related attacker groups:

[0324] Initialization: Each node represents a community;

[0325] Repeat until the modularity no longer increases:

[0326] For each node i, try to move it to the neighboring community;

[0327] Calculate the modularity gain ΔQ;

[0328] If ΔQ > 0, then move the node.

[0329] Modularity calculation formula:

[0330]

[0331] in

[0332] Step 3: Attacker fingerprint generation

[0333] For each community, generate attacker behavior fingerprints:

[0334] Fingerprint(Community_c) = {

[0335] IP_range: {the C-range of all IPs}

[0336] Time_pattern: Distribution of active time periods,

[0337] TTPs_vector: The frequency vector of TTPs within the community.

[0338] Tool_signature: An n-gram model of commonly used tools / commands.

[0339] Confidence: Community cohesion coefficient

[0340] }

[0341] Algorithm Evaluation and Adaptive Optimization:

[0342] Internal evaluation indicators:

[0343] Clustering quality assessment:

[0344] Profile coefficients (global):

[0345]

[0346] in Let i be the average distance to other points in the same cluster. Let i be the minimum average distance to the nearest other cluster;

[0347] Global contour coefficients .

[0348] Davidson-Burdin Index (Intra-cluster density):

[0349]

[0350] in Let be the average distance within cluster i. Cluster center distance

[0351] meaning Figure 1 Inconsistency index (unique to this invention):

[0352]

[0353] in Based on enhanced feature calculation

[0354] Parameter adaptive adjustment:

[0355] Dynamically adjust distance weights:

[0356] Let the current ICI be The previous round was

[0357] if :

[0358] / / meaning Figure 1 Decreased consistency necessitates increasing the weight of intent.

[0359]

[0360]

[0361]

[0362] Otherwise if :

[0363] / / meaning Figure 1 Good consistency, weight can be slightly reduced.

[0364]

[0365] Adaptive bandwidth parameter σ:

[0366] Using local scale adaptation:

[0367] For each point Calculate its up to the th Distance of neighbors ;

[0368] Local scale ;

[0369] Similarity calculation: .

[0370] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. An unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks, characterized in that, The method includes: Real-time collection of raw network logs and traffic data, followed by streaming processing and standardization; Incremental clustering algorithm is used to perform real-time clustering on the processed data, identify suspicious behavior microclusters that deviate from the normal baseline, and calculate the suspicion score of each microcluster. When the suspiciousness score of a micro-cluster exceeds a preset threshold, a deception scheme matching the suspicious behavior pattern is automatically generated, and deception resources are dynamically instantiated on a software-defined network or cloud infrastructure. Record the attacker's deep interaction sequence within the deceptive resource, including command history, system calls, file operations, and internal network connection attempts, to form high-fidelity enhanced data; The original log features are spatiotemporally aligned and fused with the enhanced data to generate an intent feature vector that characterizes the attacker's tactics, techniques and processes. Based on this intent feature vector, the suspicious micro-clusters are refined and clustered to obtain the attacker intent cluster. Within the intent cluster, an attack event attribute relationship graph is constructed through multi-level association analysis, and a community discovery algorithm is applied to identify the set of entities belonging to the same attacker or attack organization, generating an attacker behavior fingerprint.

2. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The construction of the original log features includes: extracting traffic scale features, target dispersion features, protocol and flag features, and time-series behavior features for each monitored entity within a time window; the target dispersion features include the number of unique target IPs, the number of unique target ports, target port entropy, and target IP entropy; the time-series behavior features include activity burstiness, activity period autocorrelation coefficient, connection interval statistical features, and port access sequence patterns.

3. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The feature extraction of the enhanced interactive data includes: word embedding encoding and n-gram pattern extraction of the attacker's command history; constructing a directed attribute graph from the system call sequence and file operation records, and extracting the number of nodes, number of edges, average degree, diameter and clustering coefficient of the graph; mapping the interactive behavior to the specific techniques of the MITRE ATT&CK framework, and generating binary feature vectors.

4. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The feature fusion is achieved using a self-supervised encoder: the original log features and the augmented data features are respectively input into a multilayer perceptron for dimensionality reduction, and then further fused after concatenation to output a low-dimensional intent representation vector; the encoder is jointly trained through augmented data reconstruction tasks and contrastive learning tasks to reduce the spatial distance between the intent representation vectors of the same attacker in different time windows, and increase the distance between the intent representation vectors of different attackers.

5. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The incremental clustering algorithm employs a micro-cluster maintenance mechanism: each micro-cluster maintains the linear sum and square sum of its feature vectors, the number of data points, and the last update timestamp; when a new data point arrives, its distance to the center of an existing micro-cluster is calculated; if the distance is less than the product of the micro-cluster radius and the expansion factor, the point is absorbed; otherwise, a new micro-cluster is created; micro-clusters that are too close to each other are merged periodically, and micro-clusters that have not been updated for a long time are archived and cleaned up.

6. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The refined clustering employs an intent-aware composite distance metric that integrates basic feature distance, temporal correlation distance, and intent similarity based on TTPs. The intent similarity is calculated based on whether the attacker performs the same or similar ATT&CK techniques, and the weight of intent similarity dynamically increases as the augmented data becomes richer.

7. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The multi-level correlation analysis includes: correlation at the IP and infrastructure level, including the same network segment, shared certificates or fingerprint information; correlation at the behavioral pattern level, including attack time patterns, tool usage fingerprints and error message characteristics; and correlation at the tactical, technical and process level, including the matching degree of ATT&CK tactical phases and the similarity of vulnerability exploitation methods.

8. The unsupervised clustering and source tracing method for dynamic deception-enhanced network attacks as described in claim 1, characterized in that, The method further includes: calculating the silhouette coefficient, Davidson-Bolding index, and intent consistency index of the clustering results in real time; automatically increasing the weight of intent similarity in the refined clusters when the intent consistency index decreases; appropriately decreasing the weight when the intent consistency index is good; and adjusting the bandwidth parameter of similarity calculation using a local scale adaptive method.

9. An unsupervised clustering and tracing system for dynamic deception-enhanced network attacks, characterized in that, The system includes: The data acquisition and computing module is used to collect raw logs and traffic from network devices, hosts and cloud platforms, and perform real-time streaming processing and standardization. The intelligent analysis and decision-making module includes an unsupervised clustering analysis engine and a dynamic deception decision engine. The unsupervised clustering analysis engine is used to perform real-time clustering on the processed data and identify suspicious behavior micro-clusters. The dynamic deception decision engine is used to generate deception schemes based on the suspiciousness scores of the micro-clusters. The proactive response and execution module includes a deception environment orchestrator and an enhanced data collector. The deception environment orchestrator is used to automatically deploy deception resources according to the deception scheme, and the enhanced data collector is used to record the attacker's deep interaction behavior in the deception environment and output enhanced data. The feedback loop module transmits the enhanced data back to the intelligent analysis and decision-making module to optimize the feature representation and clustering model of the unsupervised clustering analysis engine.

10. The unsupervised clustering and tracing system for dynamic deception-enhanced network attacks as described in claim 9, characterized in that, The intelligent analysis and decision-making module also includes an agent network module, which takes the original log features as input and is trained to approximate the intent representation vector output by the feature fusion encoder, thereby generating approximate intent features for network entities that have not yet triggered deception resources, for preliminary clustering and suspiciousness assessment.

Images