Attack traceability analysis system and method based on big data

Abstract

The invention discloses an attack traceability analysis system and method based on big data, and the method comprises the steps: constructing an active window traceability graph containing a cross-domain entity association relationship, introducing a self-adaptive boundary detection mechanism based on topological gravitation potential energy, intelligently screening high-value nodes, generating a causal snapshot, and building a reverse index, thereby achieving the analysis of the attack traceability. Therefore, when a threat alarm is received, historical and current data can be quickly and accurately stitched, and a complete attack causal chain which spans a network and a host and covers a long period is reconstructed. Through the mode, the accuracy of cross-domain causal association is remarkably improved, low-delay backtracking of a complex APT attack link is realized, and the integrity and reliability of traceability analysis are greatly enhanced while the real-time processing performance of the system is guaranteed, so that the overall security defense capability is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of intelligent analysis, and more specifically, to an attack tracing and analysis system and method based on big data. Background Technology

[0002] In the modern digital environment, as cyberattacks become increasingly complex and covert, especially in advanced persistent threats and industrial cyber-physical convergence scenarios, attacks often exhibit cross-device and cross-layer characteristics. To effectively address this challenge, building accurate and efficient attack attribution analysis solutions to achieve complete and real-time reconstruction of the attack chain has become one of the core requirements of cybersecurity protection systems.

[0003] Currently, existing attack attribution analysis systems in the industry typically rely on the collection and correlation analysis of multi-source logs, matching network traffic logs with host system logs using weak correlation methods such as timestamps and network 5-tuples. While these methods are effective in simple scenarios, they have significant limitations when dealing with massive amounts of heterogeneous data and large-scale distributed environments: First, network logs and host logs reflect transport layer and application layer behavior respectively, resulting in semantic gaps and making seamless logical integration difficult; second, system clocks are difficult to synchronize perfectly, and a large number of concurrent connections may exist within a time window under high concurrency, making it easy to produce ambiguity or even errors when relying solely on timestamps; furthermore, the lack of unified, deterministic correlation identifiers (such as strong correlation keys between process IDs and network connections) makes it difficult to accurately establish causal links between cross-domain entities, severely affecting the accuracy and completeness of attribution analysis.

[0004] Therefore, we look forward to an optimized attack attribution analysis method based on big data. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides an attack tracing and analysis system and method based on big data.

[0006] According to one aspect of this application, a big data-based attack tracing and analysis method is provided, comprising: S1: Perform data cleaning and entity relationship mapping on the collected raw log data stream to obtain the active window source graph; S2: Perform window boundary detection and critical asset taint assessment on the active window source graph to obtain a set of high-value boundary nodes; S3: Extract the upstream causal path summary of the high-value boundary node set in the active window source graph and package it into a causal snapshot dataset containing node identifiers, taint states and path hashes; S4: Construct an inverted index based on entity feature identifiers on the causal snapshot dataset to obtain a dormant snapshot index database; S5: In response to the received threat, trigger an alarm, extract the alarm entity identifier and perform a correlation search in the snapshot index database to obtain historical snapshot data, and logically stitch the current alarm event with the historical snapshot data to obtain a repair cause-effect path diagram; S6: Perform recursive backtracking and path integrity verification on the repaired causal path graph to obtain a complete attack source tracing report.

[0007] According to another aspect of this application, a big data-based attack tracing and analysis system is provided, comprising: The data processing module is used to clean the collected raw log data stream and map entity relationships to obtain an active window source graph. The window boundary detection and critical asset taint assessment module is used to perform window boundary detection and critical asset taint assessment on the active window traceability graph to obtain a set of high-value boundary nodes. The causal snapshot construction module is used to extract the upstream causal path summary of the high-value boundary node set in the active window source graph and package it into a causal snapshot dataset containing node identifiers, taint status and path hashes. The inverted index module is used to build an inverted index based on entity feature identifiers on the causal snapshot dataset to obtain a dormant snapshot index database; The causal path repair module is used to respond to received threats that trigger alarms, extract alarm entity identifiers and perform correlation retrieval in the snapshot index database to obtain historical snapshot data, and logically stitch the current alarm event with the historical snapshot data to obtain the causal path repair graph. The recursive backtracking and path integrity verification module is used to perform recursive backtracking and path integrity verification on the repaired causal path graph to obtain a complete attack tracing report.

[0008] Compared with existing technologies, this application provides a big data-based attack attribution analysis system and method. It constructs an active window attribution graph containing cross-domain entity relationships and introduces an adaptive boundary detection mechanism based on topological gravitational potential energy. This intelligently filters high-value nodes to generate causal snapshots and establishes an inverted index. Upon receiving threat alerts, it can quickly and accurately stitch together historical and current data to reconstruct a complete attack causal chain spanning networks and hosts and covering long periods. This significantly improves the accuracy of cross-domain causal relationships, achieves low-latency backtracking of complex APT attack chains, and greatly enhances the completeness and reliability of attribution analysis while ensuring real-time system processing performance, thereby significantly improving overall security defense capabilities. Attached Figure Description

[0009] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 This is a flowchart of an attack tracing and analysis method based on big data according to an embodiment of this application; Figure 2 This is a data flow diagram illustrating the attack tracing and analysis method based on big data according to an embodiment of this application; Figure 3 This is a flowchart of step S2 in the attack tracing and analysis method based on big data according to an embodiment of this application; Figure 4 This is a block diagram of a big data-based attack tracing and analysis system according to an embodiment of this application. Detailed Implementation

[0011] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0012] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0013] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0014] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously, as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0015] The technical solution of this application proposes an attack tracing and analysis method based on big data. Figure 1 This is a flowchart of an attack tracing and analysis method based on big data according to an embodiment of this application. Figure 2 This is a system architecture diagram of a big data-based attack tracing and analysis method according to an embodiment of this application. Figure 1 and Figure 2 As shown, the attack tracing analysis method based on big data according to an embodiment of this application includes the following steps: S1, performing data cleaning and entity relationship mapping on the collected raw log data stream to obtain an active window tracing graph; S2, performing window boundary detection and critical asset taint assessment on the active window tracing graph to obtain a high-value boundary node set; S3, extracting upstream causal path summaries of the high-value boundary node set in the active window tracing graph and packaging them into a causal snapshot dataset containing node identifiers, taint status, and path hashes; S4, constructing an inverted index based on entity feature identifiers on the causal snapshot dataset to obtain a dormant snapshot index database; S5, triggering an alarm in response to a received threat, extracting the alarm entity identifier and performing a correlation retrieval in the snapshot index database to obtain historical snapshot data, and logically stitching the current alarm event with the historical snapshot data to obtain a repair causal path graph; S6, performing recursive backtracking and path integrity verification on the repair causal path graph to obtain a complete attack tracing report.

[0016] Specifically, S1 involves data cleaning and entity relationship mapping of the collected raw log data stream to obtain an active window causal graph. The raw log data stream includes LinuxAuditd logs, network traffic logs, and web application logs. This raw data typically contains redundant fields, heterogeneous time formats, incomplete records, and unstructured text information. Directly using such raw data for correlation analysis leads to low computational efficiency, poor correlation accuracy, and difficulty in constructing a graph model that accurately reflects the causal relationships between entities. Therefore, in the technical solution of this application, the raw log data stream is preprocessed to transform the chaotic raw data into a structured, semantically clear data foundation suitable for efficient graph computation, providing a clean and clearly defined input for subsequent windowed causal graph construction. Specifically, data cleaning aims to eliminate noise and standardize data, while entity relationship mapping aims to extract entities with network security semantics (such as processes, files, network connections, and users) and their behavioral relationships (such as execution, access, and connection) from discrete log events. This is the cornerstone for constructing an active window causal graph that accurately reflects the attack chain.

[0017] In practice, the first step is to perform multi-source log normalization and cleaning on the raw log data stream to obtain a standardized cleaned log set. Specifically, this involves receiving real-time or batch log streams from different data sources (such as network probes, host proxies, and application servers). For each log format, a corresponding parser is configured, for example, using regular expressions or a dedicated log parsing library (such as a library for parsing Syslog and JSON formats) to extract key fields. The cleaning operations include: timestamp normalization (converting all timestamps to a unified UTC time format and precision, such as Unix millisecond timestamps); field extraction and renaming (extracting predefined entity identifier fields and behavior type fields from the raw logs and mapping them to unified field names); invalid data filtering (discarding log records with missing key fields, obviously incorrect formats, or belonging to known whitelist noise); and unified formatting of network identifiers such as IP addresses and port numbers. Finally, a standardized cleaned log set with consistent fields, a unified time base, and a well-organized structure is output, where each record contains core attributes such as event time, subject entity, object entity, and operation type.

[0018] Next, semantic parsing is performed on the standardized cleansing log set to transform log behaviors into a semantic structure containing subject, predicate, and object, resulting in an entity-relation triple stream. Specifically, a set of semantic parsing rules is applied to abstract the behavior described by each standardized log record into a (subject, predicate, object) triple. For example, a process creation log might be parsed as (process A, derived, process B), and a file access log might be parsed as (process C, write, file D). In this way, the linear log stream is transformed into an entity-relation triple stream with clear semantic orientation.

[0019] Furthermore, a dynamic graph structure is maintained in the in-memory database. Nodes and directed edges are created or updated by checking the existence of nodes in the entity-relation triplet flow, and nodes and edges within the window range are retained according to a sliding time window mechanism to obtain the active window source graph. Specifically, an empty directed graph structure is initialized and resides in the in-memory database. For each incoming triplet, it is checked whether its subject and object already exist as nodes in the graph. If not, a new node is created, and a directed edge labeled with the predicate is created between the subject node and the object node. Simultaneously, the system maintains a sliding time window. For nodes and edges exceeding the current window time range, the system removes them from the in-memory graph to ensure that the graph structure only contains the most recently active entities and their relationships. This dynamically maintained graph is the active window source graph.

[0020] Specifically, S2 involves performing window boundary detection and critical asset taint assessment on the active window attribution graph to obtain a set of high-value boundary nodes. It should be understood that in attack attribution analysis based on big data, especially when facing advanced persistent threats (APTs) with long latency characteristics, if the system only forcibly removes expired nodes based on simple physical time thresholds, the association between subsequent attack behaviors and their historical parent nodes will be broken, resulting in a large number of orphan nodes without context support in the memory graph. This not only compromises the accuracy of attribution analysis but also forces the system to frequently trigger high-latency cold data retrieval. Through window boundary detection and taint assessment, the system can identify critical nodes that, although temporally expired, still possess extremely high analytical value at the logical level, and provide them with special protection, thereby maintaining system throughput while ensuring real-time backtracking capabilities for complex attack paths.

[0021] Figure 3 This is a flowchart of step S2 in the attack tracing and analysis method based on big data according to an embodiment of this application. Figure 3 As shown, S2 includes: S21, performing a water level-based boundary node scan on the active window source map to obtain a set of boundary candidate nodes; S22, calculating the comprehensive taint score of each boundary candidate node in the set of boundary candidate nodes based on the active window source map to obtain a list of boundary nodes with scores; S23, performing high-value node threshold filtering on the list of boundary nodes with scores based on a preset taint retention threshold to obtain a set of high-value boundary nodes.

[0022] Specifically, in S21, a watermark-based boundary node scan is performed on the active window source map to obtain a set of candidate boundary nodes. In analysis scenarios targeting Advanced Persistent Threats (APTs), traditional hard time window truncation mechanisms pose a significant risk of context fragmentation. Specifically, in real-world attack activities, attackers often employ a strategy of using long-cycle parent nodes in conjunction with short-cycle child nodes to evade detection. For example, an attacker might create a reverse shell process as the parent node at time T0, then put that process into a long-term dormant state to evade active detection, only waking it up after 25 hours (T0+25h) and generating a new sudo command as the child node. In such scenarios, traditional mechanisms scan based solely on a single time threshold; dormant parent nodes will be judged as invalid data due to timeouts and removed from active memory (either transferred to cold storage or directly discarded). This operation directly causes the subsequent child nodes to become orphan nodes lacking contextual information in the active graph. As a result, the real-time source tracing and analysis system is unable to obtain key parent process attributes (such as the initial intrusion IP), forcing the system to initiate high-latency disk I / O operations to retrieve cold data. This severely disrupts the real-time continuity of the attack causal chain in memory, leading to a delay in threat response.

[0023] Therefore, this application proposes an adaptive boundary node screening mechanism based on topological activity feedback. Specifically, firstly, the active window source graph is initially screened based on physical time to obtain a nominal expired node set. This step aims to quickly narrow the computational scope, concentrating computational resources on edge nodes that are truly at risk of being phased out, thereby providing basic data input for subsequent in-depth evaluation while maintaining system throughput.

[0024] Next, based on the active window source graph, the subsequent topological gravitational potential energy is calculated for each node in the nominal expired node set to obtain a list of nodes with gravitational scores. Specifically, the topological gravitational potential energy formula is introduced to quantitatively evaluate the dependence of downstream active nodes on their parent node. The specific calculation logic is as follows: in, Represents a node The higher the value of the subsequent topological gravitational potential energy it bears, the more it is needed by downstream active nodes. In the active window source graph, from The set of all currently active successor child nodes reachable from the starting point; Indicates the successor child node Risk score (based on threat intelligence or anomaly detection results); This is a risk weighting coefficient used to amplify the preserving influence of high-risk child nodes on their parent nodes; This is the distance decay base (usually greater than 1), indicating that gravity decays rapidly as the number of hops in the graph increases; Indicates from node arrive The shortest path length; This is a time decay constant used to control the sensitivity of the active time of successor nodes; The current system time; child node The last active time. This means that a seemingly expired node with a high-risk, recently active direct child node will generate a huge gravitational potential energy in the calculation, indicating that the child node strongly needs the parent node's context to exist in memory, thus outputting a list of nodes with a gravity score.

[0025] Furthermore, a two-factor dynamic boundary decision is performed on the list of nodes with gravity scores to obtain a set of boundary candidate nodes. Specifically, a time aging penalty mechanism is introduced, which balances the survival time of a node with topological gravity. Only when a node expires in physical time and the topological value it provides is insufficient to offset its memory occupation cost will it be identified as a true boundary candidate. The specific decision logic can be expressed by the following formula: in, This indicates the node to be determined. This represents the final set of adaptive boundary candidate nodes. Represents a node The lifespan of the node (current time minus the last update time of the node); The preset time threshold; It is a natural logarithm function used to introduce a time aging penalty; The gravity locking threshold is a configurable parameter used to adjust the aggressiveness of the memory retention strategy. This step aims to achieve a dynamic balance between memory resources and traceability integrity. By using a logarithmic function, it ensures that the older a node is, the stronger its topological gravity needs to remain in memory. This prevents memory from being filled with zombie nodes that occupy it indefinitely, while protecting long-term parent nodes that truly provide context support. The final output is an adaptively filtered set of adaptive boundary candidate nodes.

[0026] Specifically, the embodiments of this application address the problem of long-cycle nodes being erroneously truncated due to fixed time windows in APT attack scenarios. By utilizing a topological gravitational potential energy model, it achieves on-demand retention of memory data, intelligently preserving parent nodes that, although inactive, possess interpretive power regarding current high-risk activities without significantly increasing memory pressure. This adaptive filtering strategy ensures the integrity of the causal chain in the real-time attribution graph, enabling the security analysis system to directly trace long-cycle attack paths in memory, eliminating the latency caused by cold data queries, and significantly improving the real-time perception and response capabilities to complex and persistent threats.

[0027] Specifically, in step S22, based on the active window source graph, the comprehensive taint score of each boundary candidate node in the boundary candidate node set is calculated to obtain a list of boundary nodes with scores. That is, by calculating the comprehensive taint score, the topological support value and risk weight of each node in the attack propagation path are quantitatively evaluated, thereby identifying high-value nodes that, although dormant, have interpretive power regarding current high-risk activities.

[0028] In this process, firstly, the system acquires a set of candidate boundary nodes selected through water level scanning and extracts the downstream topology of each node using the active window source graph. Then, for each candidate node, the system determines its comprehensive taint score by calculating the potential energy of the subsequent topological gravitational pull it experiences. Specifically, the system traverses all currently active successor child nodes reachable from the node in the active window source graph and accumulates their respective risk scores. During this accumulation process, the system applies a risk weighting coefficient to amplify the influence of high-risk child nodes and uses a distance decay base greater than 1 to ensure that the gravitational pull decays rapidly with the increase of the graph hop count (shortest path length). Simultaneously, a time decay constant is introduced, which is exponentially scaled based on the difference between the current system time and the last active time of the child node, thereby quantifying the logical dependence of downstream active nodes on the parent node.

[0029] Specifically, in step S23, based on a preset taint retention threshold, the list of boundary nodes with fractions is filtered for high-value nodes to obtain a set of high-value boundary nodes. That is, by filtering the list of boundary nodes with fractions for high-value nodes, parent nodes that, although inactive, have the authority to interpret current high-risk activities are intelligently identified and retained. This allows for real-time awareness of complex attack paths without significantly increasing memory pressure.

[0030] This process begins by obtaining a list of boundary nodes with fractional values ​​that have undergone preliminary scanning and gravity calculations. Next, a two-factor dynamic boundary decision logic is introduced, comparing and filtering the physical survival time of nodes with the aforementioned gravitational potential energy to determine the final set of high-value boundary nodes. Specifically, based on the node's current total survival time, a dynamically increasing aging penalty cost is constructed using the natural logarithm function, combined with a system-preset gravity locking threshold (i.e., a taint retention threshold) to define the decision boundary. Only when the topological gravitational potential energy accumulated by a node in the active graph is sufficient to offset the time aging penalty caused by its long-term memory residency, and its survival time exceeds the preset physical baseline threshold, will the node be determined to possess high-value characteristics and retained. This logic ensures that older nodes require stronger downstream active evidence to remain, achieving an intelligent balance between memory cleanup and evidence retention.

[0031] Specifically, in step S3, the upstream causal path summary of the high-value boundary node set in the active window tracing graph is extracted and packaged into a causal snapshot dataset containing node identifiers, taint states, and path hashes. It should be understood that although the high-value boundary node set has been evaluated and filtered to identify nodes requiring special handling, these nodes and their upstream causal path information still reside in the active window tracing graph in memory. Due to limited memory resources and the continuous sliding of the active window, these nodes may be lost over time or after a system restart. To achieve long-term tracing, historical querying, and offline in-depth analysis of attack activities, it is necessary to persistently store high-value nodes and their complete causal path information in a structured manner on external storage media (such as a database or file system). Therefore, in the technical solution of this application, the dynamically maintained causal chain information in memory is transformed into a persistent and queryable causal snapshot to support subsequent offline correlation analysis and historical attack chain reconstruction. By packaging it into a structured dataset containing node identifiers, taint states, and path hashes, both storage efficiency is ensured, and a foundation is provided for subsequent rapid retrieval and integrity verification.

[0032] In practice, firstly, based on the active window source graph, local causal context recursively extracts and hashes target nodes in the high-value boundary node set to obtain a list of nodes with accompanying context digests. During this process, the system starts from each high-value boundary node and recursively traverses all its upstream parent nodes along the causal edges until the root node of the source path or a preset traversal depth is reached. During the traversal, a hash is calculated for each causal path to generate a path hash value representing the uniqueness of the path structure. The path hash is typically calculated by sequentially combining the identifiers of all nodes in the path with the edge types, and then applying a hash function (such as SHA-256). This process generates a context description for each high-value boundary node that includes its own identifier, taint state, and hash digests of all its upstream paths.

[0033] Next, the list of nodes with accompanying context summaries is traversed, and the memory object properties are mapped and populated into the serialization pattern to obtain a stream of serialized snapshot objects. During this process, the system iterates through the list of nodes generated in the previous sub-step, mapping and populating the memory object properties (such as process ID, command-line arguments, network connection 5-tuple, file path, etc.) of each node according to a predefined serialization pattern (such as Protocol Buffers or Avroschema). This serialization pattern explicitly defines the fields that need to be persisted, ensuring the structural consistency and efficient read / write capabilities of the snapshot data.

[0034] Furthermore, memory graph pruning is performed on the serialized snapshot object stream to obtain a causal snapshot dataset. In the technical solution of this application, to avoid storing redundant information, a pruning operation is further performed on the serialized snapshot data to remove node attributes or path fragments that already exist in historical snapshots and have not changed, storing only the differences or referencing existing data through pointers, so as to obtain a concise causal snapshot dataset containing node identifiers, taint states, and path hashes.

[0035] Specifically, in step S4, an inverted index based on entity feature identifiers is constructed on the causal snapshot dataset to obtain a dormant snapshot index database. It should be understood that in attack tracing scenarios, when a new security event occurs, it is necessary to quickly retrieve causal path information related to the current event from massive historical snapshot data. Using a linear scanning method for querying would result in response delays that fail to meet real-time requirements. By constructing an inverted index based on entity feature identifiers, key features in the snapshot data (such as process names, file paths, IP addresses, etc.) can be mapped to a list of snapshot IDs containing those features, enabling multi-dimensional fast filtering and related queries, significantly improving retrieval efficiency. Simultaneously, setting the index database to a dormant state means that after the index is built, it enters a low-power, read-only storage mode, avoiding the consumption of system resources when there are no query requests, achieving a balance between resource optimization and long-term storage.

[0036] In practice, the system first extracts multi-dimensional feature retrieval keys from the causal snapshot dataset to obtain a feature index mapping stream. During this process, the system iterates through each snapshot object in the causal snapshot dataset, extracting predefined entity feature identifiers from the snapshot's node attributes. These feature identifiers include, but are not limited to, process names, file path hashes, network connection quintuples, user identifiers, and time window identifiers. For each extracted feature identifier, the system builds an inverted index entry, using the feature identifier as the index key, and adds the snapshot ID containing that feature to the corresponding inverted list to obtain the feature index mapping stream.

[0037] Next, a unique row key is constructed for each feature index item in the feature index mapping stream using a hash prefix and a reverse timestamp to obtain a structured storage write batch. Since the snapshot ID list in the inverted index may contain many duplicates or consecutive sequences, the system uses a compression algorithm to compress the inverted list for storage. For example, for an ordered list of snapshot IDs, difference encoding can be used to convert the list into a difference sequence, and then variable-length integer encoding can be used for compression. Simultaneously, a unique row key is generated for each index item. The row key is constructed by using the hash value of the feature identifier as a prefix and concatenating it with a reverse timestamp (i.e., the maximum timestamp minus the current timestamp) as a suffix, ensuring the uniqueness of the row key and its reverse chronological order, facilitating range queries and storage optimization. The compressed index structure not only reduces storage space but also allows for rapid reconstruction of the original list during queries using a decompression algorithm.

[0038] Next, the snapshot content records and index records from the structured storage batch are written to the data table and index table respectively to obtain the snapshot index database. That is, the compressed inverted index structure and the snapshot data itself are written to two separate tables: the data table stores the complete snapshot content records, and the index table stores the mapping records from feature identifiers to the snapshot ID list. After writing is complete, the system marks the metadata status of the index database as "dormant." The dormant state means that the index database is not currently participating in real-time query processing and exists only as read-only data. When a subsequent query request occurs, it can be loaded into memory through a wake-up mechanism. The index database in this state does not consume memory resources, but it can be restored to an active state when needed through a fast loading mechanism.

[0039] Specifically, in step S5, in response to a received threat triggering an alarm, the alarm entity identifier is extracted and correlated with the snapshot index database to obtain historical snapshot data. The current alarm event is then logically stitched together with the historical snapshot data to obtain a repair causal path graph. It should be understood that in real-time security monitoring scenarios, when suspicious behavior or threat alarms are detected, the alarm event is usually only the end of the attack chain, while the true attack source and complete path may have already moved out of the active memory window. By quickly retrieving historical snapshot data related to the alarm entity from the dormant snapshot index database and logically associating the current alarm event with these historical snapshots, the complete attack path spanning the time window can be reconstructed, revealing the initial entry point, lateral movement process, and final target of the attack, thus providing crucial evidence for incident response, threat hunting, and root cause analysis. This step solves the technical problem that traditional tracing methods cannot trace historical attack paths due to memory window limitations.

[0040] In practice, the first step is to parse threat-triggered alarms to extract a set of key entity features. Specifically, when the system receives a threat-triggered alarm (such as an EDR alarm, SIEM alarm, or a custom detection rule alarm), it first parses the alarm event and extracts key entity identifiers from the alarm payload. Alarm entity identifiers include, but are not limited to, the process ID that triggered the alarm, file path hash, network connection 5-tuple, user SID, registry keys, and other feature values ​​that can uniquely or quasi-uniquely identify a security entity. The extraction process is implemented through a predefined alarm parsing template, which defines feature extraction rules for different alarm types. For example, for process creation alarms, the process name, command-line parameters, and parent process ID are extracted; for file access alarms, the file path and access operation type are extracted. The extracted set of alarm entity identifiers is the set of key entity features, where each element represents an entity feature identifier.

[0041] Next, a multi-dimensional concurrent index retrieval is performed on the snapshot index database based on the key entity feature set to obtain the original candidate snapshot set. Specifically, each entity feature in the key entity feature set is used as a query key to retrieve data from the inverted index of the snapshot index database. During this process, it is first checked whether the snapshot index database is in a dormant state; if so, a wake-up operation is performed (loading the index data into memory). Then, for each entity feature, the corresponding snapshot ID list is searched in the inverted index. Finally, the snapshot ID lists corresponding to all entity features are combined or intersected (according to the query strategy configuration) to obtain the candidate snapshot ID set. Based on the candidate snapshot ID set, the corresponding snapshot data is read in batches from the snapshot data storage to obtain the original candidate snapshot set. It is worth noting that the retrieval process supports multi-dimensional combined queries, such as simultaneously querying process names and file paths to narrow down the candidate range.

[0042] Furthermore, the original candidate snapshot set is deserialized and causal correlation verified to obtain a list of valid historical snapshot objects. Since the retrieved candidate snapshots may contain mismatches or irrelevant data, filtering and verification are necessary. Therefore, in the technical solution of this application, the snapshots in the original candidate snapshot set are first sorted in ascending order by timestamp to ensure the correct temporal logic; then, verification is performed based on causal consistency rules, including checking whether the causal edge connections between snapshots are continuous, whether the node taint states match, and whether the path hashes are consistent. After verification, a valid historical snapshot dataset is obtained, and these snapshots are continuous in both temporal and causal logic.

[0043] Subsequently, a new graph object containing alarm nodes is initialized. The list of valid historical snapshot objects is traversed, and the restored historical snapshot nodes are added to the graph. Virtual time tunnel edges with calculated time-series decay weights are created between alarm nodes and historical snapshot nodes to obtain the repair causal path graph. In this process, the current alarm event is first encapsulated as a snapshot node, containing attributes such as alarm timestamp, entity identifier, and taint status. Then, the last snapshot node (i.e., the last node in the historical path) with the timestamp closest to the alarm time is found from the valid historical snapshot dataset. Next, a causal relationship is checked between the last snapshot node and the snapshot nodes, such as whether the process of the snapshot node was created by the process of the last snapshot node, or whether the file accessed by the snapshot node was written by the last snapshot node. If a causal relationship exists, a causal edge is established between the last snapshot node and the snapshot node, and the path hash of the snapshot node is updated to the concatenation result of the historical path hash and the current node hash. Finally, the snapshot nodes and their connected causal edges are merged with the historical snapshot data in the valid historical snapshot dataset to form a complete repair causal path graph. This pathmap spans historical windows and the current point in time, presenting the complete attack chain from the initial entry point to the final alert event.

[0044] Specifically, step S6 involves recursively backtracking and verifying the integrity of the repaired causal path graph to obtain a complete attack attribution report. It should be understood that the repaired causal path graph may have issues such as broken paths, missing nodes, or inconsistent causal logic; directly outputting it could be misleading. By using a recursive backtracking algorithm to trace back from the alarm node to the attack source, the continuity of the path can be verified, the causal relationships between nodes can be checked for reasonableness, and potentially missing intermediate links can be identified. Simultaneously, the path integrity verification mechanism verifies technical indicators such as the monotonicity of the timestamp sequence, the consistency of the path hash, and the correctness of taint propagation, ensuring the reliability of the attribution results. The final complete attack attribution report not only includes a visualized attack path graph but also structured information such as key node details, timeline analysis, attack phase divisions, and confidence assessments, providing actionable decision-making basis for subsequent emergency response, threat hunting, and root cause analysis.

[0045] In practice, the process begins by recursively backtracking and completing the causal path graph to obtain a full causal origin graph. Specifically, starting from the alarm node (i.e., the end node of the path) in the causal path graph, a depth-first traversal is performed along the reverse direction of the causal edges (i.e., from the child node to the parent node). For each visited node, it is checked whether all its parent nodes exist and are fully connected: if a parent node is missing or the type of causal edge between the parent node and the current node does not conform to the preset causal rules (such as process creation relationship, file write relationship, etc.), then that position is marked as a path breakpoint. The recursive backtracking process continues until the root node (i.e., a node without a parent node) is encountered or the preset maximum backtracking depth is reached. During the backtracking process, the system records the access status of each node, the path depth, and the location information of the path breakpoint.

[0046] Next, the attack chain confidence quantification evaluation is performed on the full causal origin graph to obtain a set of attack paths with confidence. Specifically, based on the path breakpoints and missing node information identified in the first sub-step, verification is performed in conjunction with preset path integrity verification rules. Verification rules include, but are not limited to: timestamp sequence monotonicity verification (requiring the parent node's timestamp to be less than the child node's timestamp), path hash continuity verification (requiring the child node's path hash to be an extension of the parent node's path hash), taint propagation consistency verification (requiring the parent node's taint state to propagate to the child node), and causal edge type validity verification (requiring the edge type to conform to the system-defined set of causal relationship types). For path segments that fail verification, the system will attempt intelligent completion based on context information: for example, if a process node is found to be missing a parent process node, but it is inferred from command-line parameters or file path information that it may have been created by a system process, then a virtual node will be added or marked as an unknown source; if the path hash is not continuous but the timestamp and causal logic match, then the path hash will be recalculated to restore continuity. The completed path needs to be re-verified until the integrity requirements are met or the maximum number of completion attempts is reached. Finally, a verified and complete attack path is obtained, which forms a complete causal chain from the attack source node to the alarm node.

[0047] Furthermore, based on the attack path set with confidence levels, a complete attack attribution report is generated. Specifically, the verified complete attack path is converted into a structured report that can be read and used by security analysts. The conversion process includes: converting each node in the path into a report entry, including node identifier, timestamp, entity type (process, file, network, etc.), key attributes (command line, file path, IP address, etc.), taint status, confidence score, etc.; converting causal edges between nodes into relational descriptions, including edge type, propagation direction, propagation time interval, etc.; organizing the report content in chronological order to form an attack timeline; segmenting and labeling the path according to attack stage characteristics (such as initial intrusion, lateral movement, data theft, etc.); calculating the overall path confidence score, which is based on a comprehensive calculation of factors such as path completeness, evidence sufficiency, and rule matching degree. The final output complete attack attribution report adopts a structured format (such as JSON, XML) or a visual format (such as graphical display), including complete information such as attack path details, timeline analysis, attack stage division, confidence assessment, and recommended response measures.

[0048] In summary, the attack attribution analysis method based on big data according to the embodiments of this application is explained. It constructs an active window attribution graph containing cross-domain entity relationships and introduces an adaptive boundary detection mechanism based on topological gravitational potential energy. This intelligently filters high-value nodes to generate causal snapshots and establishes an inverted index, enabling rapid and accurate stitching of historical and current data upon receiving threat alerts. This reconstructs a complete attack causal chain spanning networks and hosts and covering long periods. In this way, the accuracy of cross-domain causal relationships is significantly improved, achieving low-latency backtracking of complex APT attack chains. While ensuring the real-time processing performance of the system, it greatly enhances the completeness and reliability of attribution analysis, thereby significantly improving overall security defense capabilities.

[0049] Furthermore, an attack tracing and analysis system based on big data is also provided.

[0050] Figure 4 This is a block diagram of a big data-based attack tracing and analysis system according to an embodiment of this application. Figure 4As shown, the attack tracing and analysis system 300 based on big data according to an embodiment of this application includes: a data processing module 310, used to perform data cleaning and entity relationship mapping on the collected raw log data stream to obtain an active window tracing graph; a window boundary detection and critical asset taint assessment module 320, used to perform window boundary detection and critical asset taint assessment on the active window tracing graph to obtain a high-value boundary node set; and a causal snapshot construction module 330, used to extract the upstream causal path summary of the high-value boundary node set in the active window tracing graph and package it into a causal snapshot containing node identifier, taint status, and path hash. The system includes: a snapshot dataset; an inverted index module 340, used to construct an inverted index based on entity feature identifiers on the causal snapshot dataset to obtain a dormant snapshot index database; a repair causal path stitching module 350, used to respond to received threats that trigger alarms, extract alarm entity identifiers and perform correlation retrieval in the snapshot index database to obtain historical snapshot data, and logically stitch the current alarm event with the historical snapshot data to obtain a repair causal path graph; and a recursive backtracking and path integrity verification module 360, used to perform recursive backtracking and path integrity verification on the repair causal path graph to obtain a complete attack tracing report.

[0051] As described above, the big data-based attack tracing and analysis system 300 according to embodiments of this application can be implemented in various wireless terminals, such as servers with big data-based attack tracing and analysis algorithms. In one possible implementation, the big data-based attack tracing and analysis system 300 according to embodiments of this application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the big data-based attack tracing and analysis system 300 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the big data-based attack tracing and analysis system 300 can also be one of many hardware modules of the wireless terminal.

[0052] Alternatively, in another example, the big data-based attack tracing and analysis system 300 and the wireless terminal can also be separate devices, and the big data-based attack tracing and analysis system 300 can be connected to the wireless terminal via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0053] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for attack tracing and analysis based on big data, characterized in that, include: S1: Perform data cleaning and entity relationship mapping on the collected raw log data stream to obtain the active window source graph; S2: Perform window boundary detection and critical asset taint assessment on the active window source graph to obtain a set of high-value boundary nodes; S3: Extract the upstream causal path summary of the high-value boundary node set in the active window source graph and package it into a causal snapshot dataset containing node identifiers, taint states and path hashes; S4: Construct an inverted index based on entity feature identifiers on the causal snapshot dataset to obtain a dormant snapshot index database; S5: In response to the received threat, trigger an alarm, extract the alarm entity identifier and perform a correlation search in the snapshot index database to obtain historical snapshot data, and logically stitch the current alarm event with the historical snapshot data to obtain a repair cause-effect path diagram; S6: Perform recursive backtracking and path integrity verification on the repaired causal path graph to obtain a complete attack source tracing report.

2. The attack tracing and analysis method based on big data according to claim 1, characterized in that, The raw log data stream includes LinuxAuditd logs, network traffic logs, and web application logs.

3. The attack tracing and analysis method based on big data according to claim 1, characterized in that, Step S1 includes: The raw log data stream is cleaned by multi-source log normalization to obtain a standardized cleaned log set; Semantic parsing is performed on the standardized cleaning log set to transform log behavior into a semantic structure containing subject, predicate, and object to obtain entity relation triple flow; The dynamic graph structure is maintained in the in-memory database. Nodes and directed edges are created or updated by checking the existence of nodes in the entity relation triplet flow. Nodes and edges within the window range are retained according to the sliding time window mechanism to obtain the active window source graph.

4. The attack tracing and analysis method based on big data according to claim 1, characterized in that, Step S2 includes: A water level-based boundary node scan is performed on the active window source map to obtain a set of boundary candidate nodes; Based on the active window source graph, the comprehensive taint score of each boundary candidate node in the boundary candidate node set is calculated to obtain a list of boundary nodes with scores. Based on a preset taint retention threshold, the list of boundary nodes with scores is filtered by a high-value node threshold to obtain a set of high-value boundary nodes.

5. The attack tracing and analysis method based on big data according to claim 1, characterized in that, Step S3 includes: Based on the active window source graph, local causal context recursive extraction and hash calculation are performed on target nodes in the high-value boundary node set to obtain a list of nodes with context summaries. Iterate through the list of nodes with context summaries and populate the serialization pattern with memory object property maps to obtain a stream of serialized snapshot objects; Memory graph pruning is performed on the serialized snapshot object stream to obtain a causal snapshot dataset.

6. The attack tracing and analysis method based on big data according to claim 1, characterized in that, Step S4 includes: Multidimensional feature retrieval key extraction is performed on the causal snapshot dataset to obtain the feature index mapping stream; A unique row key is constructed from the feature index items in the feature index mapping stream using a hash prefix and a reverse timestamp to obtain the structured storage write batch; The snapshot content records and index records in the structured storage batch are written to the data table and index table respectively to obtain the snapshot index database.

7. The attack tracing and analysis method based on big data according to claim 1, characterized in that, Step S5 includes: Analyze threat-triggered alerts to extract key entity feature sets; Multidimensional indexing and concurrent retrieval are performed on the snapshot index database based on the key entity feature set to obtain the original candidate snapshot set; Snapshot deserialization and causal correlation verification are performed on the original candidate snapshot set to obtain a list of valid historical snapshot objects; Initialize a new graph object containing alarm nodes, traverse the list of valid historical snapshot objects and add the restored historical snapshot nodes to the graph, and create virtual time tunnel edges with calculated time decay weights between alarm nodes and historical snapshot nodes to obtain the repair causal path graph.

8. The attack tracing and analysis method based on big data according to claim 1, characterized in that, Step S6 includes: Recursive deep backtracking and graph completion are performed on the repaired causal path graph to obtain a full causal source graph. A quantitative evaluation of attack chain confidence is performed on the full causal source graph to obtain a set of attack paths with confidence. A complete attack attribution report is generated based on the attack path set with confidence levels.

9. The attack tracing and analysis method based on big data according to claim 4, characterized in that, A waterline-based boundary node scan is performed on the active window source map to obtain a set of candidate boundary nodes, including: Perform a preliminary screening of the active window source graph based on physical time to obtain the nominal expired node set; Based on the active window tracing graph, the subsequent topological gravitational potential energy of each node in the nominal expired node set is calculated to obtain a list of nodes with gravitational fractions. A two-factor dynamic boundary decision is performed on the list of nodes with gravity scores to obtain a set of candidate boundary nodes.

10. An attack tracing and analysis system based on big data, characterized in that, include: The data processing module is used to clean the collected raw log data stream and map entity relationships to obtain an active window source graph. The window boundary detection and critical asset taint assessment module is used to perform window boundary detection and critical asset taint assessment on the active window traceability graph to obtain a set of high-value boundary nodes. The causal snapshot construction module is used to extract the upstream causal path summary of the high-value boundary node set in the active window source graph and package it into a causal snapshot dataset containing node identifiers, taint status and path hashes. The inverted index module is used to build an inverted index based on entity feature identifiers on the causal snapshot dataset to obtain a dormant snapshot index database; The causal path repair module is used to respond to received threats that trigger alarms, extract alarm entity identifiers and perform correlation retrieval in the snapshot index database to obtain historical snapshot data, and logically stitch the current alarm event with the historical snapshot data to obtain the causal path repair graph. The recursive backtracking and path integrity verification module is used to perform recursive backtracking and path integrity verification on the repaired causal path graph to obtain a complete attack tracing report.

Images