Big data risk early warning and evaluation method based on artificial intelligence

Abstract

The invention relates to the field of internet security, and discloses a big data risk early warning and evaluation method based on artificial intelligence, comprising the following steps: step S1, collecting original security data from a heterogeneous data source; s2, constructing a global causal model; s3, constructing and evolving an event causal evolution diagram so as to establish directed edges with weights among the nodes; s4, dynamically evaluating the ability level of the attacker; s5, performing adversarial intention projection; s6, calculating a dynamic risk score; and generating an early warning when the score exceeds an early warning threshold. According to the method, asymmetric information flows among event types are quantified through transfer entropy, and a context evidence fusion mechanism is combined, so that a real causal relationship and a simple statistical correlation can be distinguished; the defect that a high false alarm rate is easily generated based on rule or simple threshold matching is overcome, so that the event causal evolution diagram can accurately reflect the internal logic and time sequence characteristics of an attack behavior, and the accuracy of complex threat perception is improved.

Description

Technical Field

[0001] This invention relates to the field of internet security, specifically to a big data risk warning and assessment method based on artificial intelligence. Background Technology

[0002] As digital transformation deepens, enterprise network environments are becoming increasingly complex, and cybersecurity threats are characterized by high concealment, long duration, and diverse attack methods. In particular, advanced persistent threats (APTs) have become a major risk faced by organizations. To address these challenges, Security Operations Centers (SOCs) typically deploy a large number of cybersecurity monitoring devices and log auditing systems to collect massive amounts of heterogeneous security data and attempt to identify potential intrusion behaviors through correlation analysis techniques.

[0003] However, existing security data analysis and risk assessment technologies still have significant limitations in practical applications. First, traditional big data analysis methods often rely on predefined rule matching or simple statistical correlation analysis to detect anomalies. This correlation-based analysis often ignores the deeper causal logic between events, making it difficult to distinguish the true causal chain of attack behavior from random coincidences in the background noise, resulting in a large number of alerts and a high false alarm rate. Especially when facing complex attacks spanning long time windows or involving multiple entities, the lack of refined consideration of entity context and temporal patterns makes it difficult for the analysis system to construct an accurate chain of evidence reflecting the attack's evolution.

[0004] Furthermore, existing technologies for predicting and assessing attack behavior typically rely on static attack graphs or generic probabilistic models. These methods often assume attackers possess unlimited resources or a constant level of capability, failing to consider the heterogeneity and dynamic changes in attacker capabilities in real-world adversarial scenarios. When predicting attack paths, the lack of dynamic constraints on the attacker's current skill level and resource status leads the system to traverse all theoretically possible attack paths, resulting in a combinatorial explosion of calculations. This includes numerous paths that are impossible under actual capability constraints, thus reducing the practical guidance and computational efficiency of risk warnings.

[0005] Finally, in the defense response phase, existing risk assessment systems typically stop at generating alerts, lacking the ability to proactively assess the effectiveness of defense strategies. When faced with high-risk warnings, security operations personnel often have to rely solely on personal experience or contingency plans to select response measures, finding it difficult to predict the specific quantifiable effects of different defense strategies on reducing future risks before taking action. This lack of closed-loop feedback and decision support often leaves defense responses in a passive and trial-and-error state, making it difficult to achieve data-driven optimal defense decisions. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a big data risk warning and assessment method based on artificial intelligence. This method solves the problems of high false alarm rates due to reliance on simple correlation analysis and lack of deep causal logic, unrealistic risk extrapolation due to lack of perception and constraint on the dynamic changes in attacker capabilities, and insufficient decision support due to lack of forward-looking quantitative assessment of the effectiveness of defense strategies.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a big data risk warning and assessment method based on artificial intelligence, comprising the following steps: First, data acquisition and standardization. Raw security data is collected from heterogeneous data sources such as the network layer, host layer, and application layer, and then parsed and converted into a standardized atomic event stream. Each atomic event... It is defined as a structure containing a unique event identifier, an occurrence timestamp, a standardized event type, a source entity, a target entity, and a feature vector extracted from the payload.

[0008] Secondly, a global causal model based on transfer entropy is constructed. To characterize the inherent logic between events rather than simple correlations, this method utilizes transfer entropy to quantify the asymmetric information flow between different event types. For any two event types... and Calculate from arrive The transfer entropy The formula is as follows: ; In the formula, Indicates event type exist The state at any given moment (e.g., occurred or did not occur); Indicates event type At any moment Previous The vector composed of historical states is ; Indicates event type At any moment Previous A vector composed of historical states, i.e. ; This represents the joint or conditional probability obtained statistically from historical data. When... When the causal confidence threshold is exceeded, the causal link is confirmed as valid, and its conditional transition probability and typical average time delay are recorded. and delay variance This allows for the construction of a global causal model.

[0009] Third, an event causal evolution graph is constructed and evolved. Based on the real-time received atomic event stream, a directed acyclic graph is dynamically constructed. When determining the connections between nodes, this method relies not only on the basic confidence level provided by the global causal model. It also introduces contextual evidence scoring. To quantify the strength of association between specific event instances in terms of entity and time dimensions. The calculation formula is: In the formula, , These are the cause node and the result node, respectively. For entity similarity function, These are the weight hyperparameters. Subsequently, the final edge weights are determined through a nonlinear evidence fusion mechanism. : ; This fusion mechanism ensures that the determination of causal relationships conforms to both global statistical laws and local contextual features.

[0010] Fourth, dynamically assess the attacker's capability level. When an attack chain forms in the diagram... In this method, the attacker's resource and technical capabilities are inferred in real time based on the execution cost of each attack action during the attack chain evolution. Attacker Capability Level Adjustments will be made based on the following dynamic update logic: ; In the formula, For the first Pre-step ability level, For the first The cost of executing an attack action. This is the learning rate factor. This mechanism enables the present invention to adaptively improve the threat level assessment based on newly observed high-cost behaviors.

[0011] Fifth, perform adversarial intent projection. Starting from the latest node in the attack chain, use Monte Carlo simulation to generate a set of future attack paths. During this process, the attacker capability level obtained from the above assessment will be... The dynamic threshold for path pruning: if the cumulative cost of a simulated path exceeds... If the path is deemed beyond the attacker's capabilities, it is terminated. Simultaneously, in conjunction with defensive perturbation constraints, the probability of paths associated with already executed defensive actions is set to zero. Through extensive simulations, the evolution of the attack chain to the intent states of each terminal is calculated. conditional probability and expected arrival time .

[0012] Sixth, calculate the dynamic risk score. Based on the prediction results, use a dynamic risk calculation function to quantify the urgency and potential harm of the current threat. Risk Score The calculation formula is: ; In the formula, The service impact value of the terminal intent state. This is a time penalty function (such as a logarithmic function) based on the expected arrival time, used to amplify the impact of time urgency on risk.

[0013] Finally, decision support based on the expected risk reduction value is provided. For the set of alternative defensive actions, the method constructs a temporary, perturbed graph copy in memory and re-executes the adversarial intent projection and risk calculation to obtain the future risk score. Based on this, each defensive action is calculated. Expected risk reduction value : ; in accordance with The system prioritizes and recommends defensive actions, and after an action is implemented, it automatically corrects the graph structure based on feedback from intervention events, triggering a new round of risk assessment, thus forming a closed-loop dynamic defense system.

[0014] This invention provides a big data risk early warning and assessment method based on artificial intelligence. It has the following beneficial effects: 1. This invention utilizes transfer entropy to quantify the asymmetric information flow between event types and combines it with a contextual evidence fusion mechanism that includes entity correlation and temporal pattern similarity. This allows it to distinguish between true causal relationships and simple statistical correlations. Furthermore, it overcomes the shortcomings of traditional rule-based or simple threshold matching, which easily leads to high false alarm rates. This enables the constructed event causal evolution graph to more accurately reflect the internal logic and temporal characteristics of attack behavior, thereby improving the accuracy of perception of complex threats.

[0015] 2. This invention applies a dynamic update logic to the attacker's capability level and uses this level as a dynamic path pruning constraint in the process of projecting adversarial intent. This makes risk prediction not based on static assumptions, but can adaptively adjust the scope of the inference according to the actual technical complexity and resource consumption exhibited by the attacker. This can eliminate attack paths that exist in theory but are not feasible under the current capability constraints, thereby improving the computational efficiency and realistic credibility of forward-looking risk inference.

[0016] 3. This invention provides a decision support and closed-loop response mechanism based on the calculation of expected risk reduction value. By pre-simulating the impact of different defense actions on dynamic risk scores in a virtual graph replica and feeding back the actual intervention events to the model for causal redirection, it can achieve quantitative evaluation and optimal recommendation of the effectiveness of defense strategies, transforming security operations from passive response to data-driven proactive defense, and improving the pertinence and decision quality of emergency response. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 The flowchart for constructing the global causal model based on transfer entropy in this invention is shown below. Figure 3 This is a schematic diagram illustrating the dynamic update principle of attacker capability levels in this invention. Figure 4 This is a schematic diagram of the adversarial intent projection deduction process based on capability constraints according to the present invention; Figure 5 This is the logic diagram of dynamic risk calculation and time penalty effect of the present invention; Figure 6 This is a flowchart of the defensive decision-making closed loop based on the expected risk reduction value of the present invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To better understand the present invention, the above content will be described in detail below with reference to specific embodiments.

[0020] Please see the appendix Figure 1 - Appendix Figure 6 This invention provides a big data risk warning and assessment method based on artificial intelligence, which includes the following steps: Step S1: Collect raw security data from heterogeneous data sources; Step S2: Construct a global causal model; Step S3: Construct and evolve an event causal evolution graph; Step S4: Dynamically assess the attacker's capability level; Step S5: Perform adversarial intent projection; Step S6: Calculate a dynamic risk score.

[0021] In one embodiment, the method of the present invention first collects raw security data from one or more heterogeneous data sources. The data sources cover multiple layers of the enterprise IT environment, specifically including network layer devices such as firewalls, routers, and switches; security protection devices (such as Intrusion Detection Systems (IDS), Intrusion Prevention Systems (IPS), and Web Application Firewalls (WAF); host layer endpoints such as servers and personal computers, obtained through endpoint detection and response (EDR) agents or operating system logs; application layer services such as database access logs, web server access logs, and cloud platform audit logs (such as AWS). The data collection method can vary depending on the characteristics of the data source, employing techniques well-known to those skilled in the art, including but not limited to Syslog protocol push, periodic polling via application programming interfaces (APIs), or subscription to message queues (such as Kafka and RabbitMQ).

[0022] The collected, diverse, semi-structured, or unstructured raw log data is processed using a set of configured parsing rules to extract key information fields. These parsing rules are predefined for specific log formats; for example, regular expressions can be used to match and extract key fields, or dedicated parsers can be used for specific devices, such as Cisco-ASA firewalls and Apache web servers. This transforms text-based log entries into machine-readable structured data.

[0023] After parsing and extraction, each original log entry is transformed and mapped to a standardized data unit, namely an atomic event. In an embodiment of the present invention, an atomic event... It is defined as a unified structure, and its formula is: ; In the formula: This is a unique identifier for the event. To ensure the uniqueness of each atomic event within the system, this identifier can be generated by hashing the contents of the original log entry and its received timestamp (e.g., using SHA-256). This refers to the timestamp of the event. The method of this invention can parse time information from the raw logs and convert it into Coordinated Universal Time (UTC) format to eliminate time inconsistencies caused by devices being located in different time zones, providing a standard time reference for subsequent causal relationship and time series analysis.

[0024] This involves standardized event types. This is the foundation for event standardization, and the method internally maintains a standardized, hierarchical event type classification system. This system can be built based on industry-recognized attack frameworks or customized according to specific business scenarios. During parsing, event names extracted from the raw logs, such as Port-Scan-detected and Failed-login-attempt, are mapped to a standard type within this classification system, such as Recon.Network.PortScan and Auth.Login.Failure. This process unifies semantically similar but differently named events reported by different vendors and devices into the same type within this standardized event type classification system, thus resolving the issue of heterogeneous event semantics.

[0025] and These represent the source entity and the target entity of the event, respectively. Since these entities are participants in network behavior and exhibit diverse forms, the method of this invention abstracts them uniformly into entity objects. Specifically, an entity can be a network address, such as an IP address or MAC address; a user account, such as a domain name or local account name; a host or device, such as a hostname or device ID; or a process or file (e.g., process name or file path). Therefore, in this way, the method of this invention can process and associate activities at different levels.

[0026] This is a feature vector extracted from the event payload. In addition to basic metadata, raw logs also contain contextual information, i.e., the payload. This method extracts features from this payload, transforming it into a fixed-dimensional numerical vector for subsequent processing by machine learning models. Specific implementation methods include: for text-based payloads (such as URLs and SQL queries), keyword features can be extracted using methods such as the bag-of-words model and TF-IDF, or semantic embedding vectors can be generated using pre-trained language models (such as BERT); for specific metrics (such as file hash values ​​MD5 / SHA256, domain names), they are directly used as classification features or unique identifiers.

[0027] Therefore, through the above steps, raw security logs from different sources and in different formats are continuously transformed into a homogeneous, structured, and semantically rich stream of atomic events.

[0028] After completing the standardization process of the raw data to obtain a unified atomic event flow, this method moves on to the next step in the offline modeling stage: learning from the data and building a global causal model that can reveal the internal logic of attack behavior.

[0029] Specifically, to achieve this goal, this method employs transfer entropy from information theory to quantify the asymmetric information flow between different event types, in order to distinguish between genuine causal relationships and simple co-occurrence or correlation. Specifically, for any two event types determined during the standardization process, such as type... and type First, from the massive historical atomic event data, their discrete time series arranged in chronological order are extracted. Based on this, calculations are performed from... arrive The transfer entropy It is used to measure in the known Given its own historical information, Historical information can also be used for prediction How much additional information does the future state provide? The formula for calculation is as follows: ; In the formula, Indicates event type exist The state at any given moment (e.g., occurred or did not occur); Indicates event type At any moment Previous The vector composed of historical states is ; Indicates event type At any moment Previous A vector composed of historical states, i.e. ; This represents the joint or conditional probability statistically derived from historical data. For calculating this probability, those skilled in the art can employ various methods, including frequency-based histogram statistics or kernel density estimation; the specific implementations of these methods are well-known in the field and will not be elaborated upon here.

[0030] The aforementioned transfer entropy is calculated by applying it to all preset event types. When the calculated... The value is significantly greater than zero and exceeds a predefined causal confidence threshold. Then, it is considered from the event type To event type There exists a potential, statistically significant causal link. For each confirmed causal link, its corresponding conditional transition probability will be further statistically analyzed and recorded. Typical average time delay and the statistical variance of this delay .

[0031] Ultimately, all confirmed causal links and their associated quantified parameters are stored in a structured manner, collectively forming a Global Causal Model (GCM). This model is essentially a knowledge graph, where nodes represent standardized event types and edges represent causal relationships learned from the data, with probabilistic and temporal attributes.

[0032] In parallel with the aforementioned data-driven GCM construction process, this method also requires the construction of two knowledge bases based on expert knowledge to support subsequent attacker capability assessment and risk quantification.

[0033] The first knowledge base is an attack cost model, used for each standardized attack behavior. (Typically corresponding to a causal edge or an event type in GCM) Assign a quantified execution cost. The execution cost This can be a numerical value used to measure the technical complexity and resource consumption required to carry out the attack. Its value can be determined by considering one or more of the following factors: the difficulty of obtaining the attack tools (e.g., publicly available script tools are low-cost, while specially developed zero-day exploit tools are high-cost), the complexity of the required preconditions, and the inherent risk of the attack being detected by the security system. In practical implementation, each technology or sub-technology (ID) in the MITRE-ATT&CK framework can be mapped to a pre-defined cost score, forming a key-value pair storage that can be queried by the system.

[0034] The second knowledge base is the endpoint intent state base. It defines a series of final target states that attackers might pursue, i.e., endpoint intent states. These states represent various security incidents that pose a serious threat to the organization, for example, This represents a core database data breach. This means that the domain controller permissions are fully controlled. This indicates that a critical business system has been encrypted by ransomware. For each endpoint intent state in this database... Each of them is assigned a quantified business impact value. The impact value It is a numerical value used to measure the extent of damage that would be caused to an organization's business operations, financial condition, or reputation should that intended state be achieved. Its value is typically determined through a business impact analysis, jointly assessed by experts from information security, IT operations, and relevant business units.

[0035] These two knowledge bases enable the transformation of abstract security concepts (such as attack difficulty and business risk) into quantifiable data that can be processed by machines.

[0036] After the global causal model and related knowledge base are constructed, this method enters the online analysis and response phase, which is used to construct and evolve an event causal evolution graph (ECEG) that can reveal the internal logic of the attack based on the atomic events received in real time.

[0037] When a standardized new atomic event Upon acceptance, it is first added to ECEG as a new node. Subsequently, methods that already exist in the graph and whose timestamps are earlier than [previous timestamp] are added. Search for potential cause nodes among the nodes. For each pair of candidate nodes The method first queries the Global Causal Model (GCM) built in the offline phase. If the GCM records information from... event types arrive event types If a causal link is identified, the method extracts the corresponding conditional transition probability from it, which serves as the basic confidence level that a causal relationship exists between the two. The baseline confidence level It is a universal probability derived from historical data statistics, representing the general correlation strength between event types.

[0038] Relying solely on the basic confidence level between event types This approach fails to fully utilize the rich contextual information contained in specific event instances. Therefore, this invention introduces a contextual evidence scoring system. The calculation is used to quantify two specific event instances. and The strength of association at the contextual level. This score combines evidence from two dimensions: entity association and temporal pattern similarity. The formula is as follows: ; In the formula, It is an entity similarity function with a range of [0,1], used to measure the similarity of events. and Participating entities (source entities) or target entity The degree of correlation between () and (). In one specific implementation, the function can be determined by checking (). and The calculation is based on whether the source and destination entity fields meet one or more of the following conditions: (Same attack source) (Same attack target), or (Forming an attack springboard). The more conditions are met, the better. The higher the value, the better.

[0039] The second term in the formula is a Gaussian kernel function, used to measure the actual time interval between two events. Compared to the typical average time delay of this causal link recorded in GCM The degree of similarity. This is the statistical variance of the delay obtained from GCM. The closer the actual time interval is to the typical delay, the closer this term is to 1; conversely, it is closer to 0.

[0040] It is a preset weight hyperparameter with a value between [0,1], used to adjust the relative importance of entity relevance and temporal pattern similarity in calculating the total contextual evidence score.

[0041] After obtaining global base confidence scores based on event types, respectively and contextual evidence scores based on event instances Subsequently, this method employs a nonlinear evidence fusion mechanism to combine the two pieces of evidence from different dimensions to generate a final, more reliable causal edge weight. The fusion process aims to achieve the following: when the confidence level of any one source of evidence is high, the final fusion confidence level will also be high; only when the confidence levels of both sources of evidence are low will the final confidence level be low. The evidence fusion formula used in this embodiment is as follows: ; Its calculation results This is the final causal confidence score that integrates global statistical regularities and current local contextual evidence. Finally, the method calculates the final causal confidence score... With predefined confirmation threshold Compare. If Greater than Then at node and A directed edge is formally created between them, and This serves as the weight for that edge. Simultaneously, the method queries the attack cost model for events... Corresponding attack cost This is then added as another attribute of that edge. Therefore, this method can connect isolated events in real time and accurately into a dynamically evolving event causal evolution graph with quantified confidence and cost attributes, providing a graph structure foundation for subsequent attacker capability assessment and intent deduction.

[0042] After the initial formation of the dynamic graph structure of the Event Causal Evolution Graph (ECEG), this method further analyzes the attack chain in the graph to infer the attacker's capabilities behind the attack behavior, thereby providing a key constraint that conforms to real-world logic for subsequent predictions.

[0043] An attack chain occurs when one or more nodes in an ECEG form a connected subgraph through edges with causal confidence. In this case, this method will assign and dynamically maintain an attacker capability level for the attack chain. .Should This is a numerical value used to quantify the technical complexity and resource ceiling possessed by the attacker initiating the attack chain. It is used when the attack chain is first identified, for example, when two event nodes pass through the first chain with a cost of... When connecting edges, an initial capability level is assigned to the chain. In one specific implementation, this initial capability level... It can be directly set as the cost of the first observed attack behavior, i.e. .

[0044] As new atomic events are continuously added, the attack chain... It will continue to evolve and expand. In the method of this invention, the attacker's capability level... It is not a static initial value, but rather a dynamically updated adaptive value based on newly observed attack behaviors. When the attack chain... Along a cost of The edge expansion is completed, thus completing the first... Each attack action corresponds to its corresponding ability level. The following dynamic capability update formula will be used for adjustment: ; In the formula, Indicates that when the first... After each attack action, the attacker's ability level is updated; Indicates that when the first... The ability level prior to each attack action; Indicates the first The execution cost of each attack action, which is derived from the cost attribute of the corresponding edge in ECEG. Obtain; This is a preset learning rate factor with a value range of [0,1], used to control the speed at which the ability level assessment responds to new evidence. A larger [value]... The value indicates that the evaluation results will more quickly approximate the most recently observed high-cost behaviors, while smaller values ​​will... The value makes the evaluation results smoother and more resistant to noise from a single abnormally high-cost event.

[0045] The underlying logic of this formula is that, This represents an assessment of the highest technical cost possessed by the attacker. Therefore, only when the cost of a newly observed attack action is considered... Exceeding the current competency assessment Only then is it considered that new information about the attacker's capabilities has been obtained, and the capability level is upgraded accordingly. The extent of the upgrade is determined by the excess portion. and learning rate The decision is made jointly. If the cost of the new action does not exceed the current assessment, the capability level remains unchanged. Therefore, in this way, It can become a non-decreasing, dynamic indicator that reflects the highest level of skill that the attacker has demonstrated.

[0046] Ultimately, for each attack chain Dynamically calculated ability level These will all serve as key constraints, passed on to subsequent projections of adversarial intentions. Specifically, This is a dynamic threshold used for path pruning in adversarial intent projection. When adversarial intent projection explores future attack paths through Monte Carlo simulation, for any simulated path being constructed, this invention accumulates the cost of all virtual attack actions along that path in real time. Once the accumulated cost of a simulated path exceeds the capability level corresponding to its source attack chain... The simulated path will be determined to be beyond the attacker's capabilities and will be immediately terminated (i.e. pruned). At this point, the invention will no longer continue to explore along that path. Therefore, by applying this constraint, the intent deduction process can eliminate attack sequences that are logically and resource-impractical, and concentrate computational resources on analyzing more likely threats, thereby improving the accuracy and practical significance of the final risk prediction.

[0047] After dynamically assessing the attacker's capability level for the attack chain, this capability level can be used as input to guide the forward-looking risk extrapolation process of this method. This process can be based on adversarial intent projection to simulate and predict the future evolution path of the identified attack chain, and finally convert the prediction results into a quantifiable risk score.

[0048] Specifically, the adversarial intent projection is based on Monte Carlo simulation, specifically, for each active attack chain... To make forward-looking predictions about the future trend, based on the attack chain The latest event node with the middle timestamp As a starting point for the extrapolation, a set of future attack paths is generated by performing a large number of constrained forward stochastic simulations. In each independent simulation, the generation of virtual attack paths follows these steps: First, during the path extension phase, starting from the end node of the current simulated path, the Global Causality Model (GCM) is queried to obtain all subsequent event types with causal relationships originating from the event type of that end node, along with their corresponding conditional transition probabilities. At this point, the adversarial intent projection performs weighted random sampling based on these probabilities, selecting a subsequent event type to generate a virtual subsequent event node, thus extending the simulated path forward by one step.

[0049] Secondly, during the path expansion process, this method integrates and applies two dynamic constraints to ensure the realism and effectiveness of the simulation. The first is the capability level constraint. After each step of the simulated path expansion, the cost of all virtual attack steps on that simulated path is accumulated in real time. The accumulated cost is correlated with the dynamic attacker capability level corresponding to the source attack chain S of that path. Compare. If the cumulative cost exceeds... If the simulated path fails to meet the target, it means that the resource consumption of the attack sequence represented by this simulated path has exceeded the current assessment limit of the attacker's capabilities. Therefore, this simulated path is deemed invalid and immediately terminated, and the invention will not proceed with further exploration along this branch. The second constraint is the defense perturbation constraint. The simulation is performed based on the ECEG topology modified by the intervention event. If the security team has already implemented defensive actions (e.g., blocking an IP address), the transition probability of all potential causal paths related to the blocked entity is considered zero in this simulation. This allows adversarial intent projection to automatically avoid attack paths that have been cut off by defensive measures.

[0050] Each simulated path generation process will terminate when one of the following conditions is met: the end node of the path reaches a predefined terminal intent state in the terminal intent state library. Or the path length has reached a preset maximum simulation depth to prevent infinite loops. After executing a preset number of (e.g., After (number of) simulations, the adversarial intention projection statistics are... In this simulation, the final intention state of each terminal is reached. Number of paths This allows for the calculation of the current attack chain. Evolved to conditional probability At the same time, record all successful arrivals. The average time span of the path is used to obtain the expected arrival time. .

[0051] The predictive results based on the projection output of adversarial intent provide direct data input for the final dynamic risk quantification. This method mathematically integrates three dimensions—the probability of intent occurrence, its potential business impact, and the urgency of the threat—through a dynamic risk calculation function, thereby calculating a comprehensive and forward-looking risk score. For each active attack chain… At any given moment Dynamic risk score The following formula is given, and its expression is: ; In the formula, Representative attack chain At any moment Dynamic risk scoring; Represents the output projected from an adversarial intent, from the attack chain. At any moment Starting from the state, it eventually evolves into the terminal intent state. The conditional probability; This represents the terminal intent state obtained from the terminal intent state library. The quantitative impact on business after the achievement; This represents the output from the adversarial intent projection, indicating the transition from the current state to the terminal intent state. Expected time; This is a time penalty function, a monotonically increasing function, used to amplify the impact of time urgency on risk scoring. A specific implementation method is to use a logarithmic function, for example... ,in For the expected time Therefore, it can be seen that the function can make the denominator smaller as the expected arrival time is shorter, thus resulting in a higher risk score.

[0052] The final calculated dynamic risk score This is a quantitative indicator that comprehensively reflects the likelihood of a threat, the severity of its harm, and the urgency of the situation. When this score exceeds a pre-set warning threshold, a detailed risk warning will be generated, providing a basis for subsequent defense decisions.

[0053] After calculating the dynamic risks of the attack chain, this method not only outputs early warnings, but also provides a closed-loop response and evaluation mechanism to transform externally executed defense strategies into understandable inputs and simulate their impact on the attack path at the model level, thereby achieving dynamic redirection of causal inference.

[0054] Specifically, when security operations personnel execute a defensive action based on risk warnings or their own judgment, this action is input into this method through a pre-defined interface and processed as an intervention event. Structurally, an intervention event is defined as a data unit containing an action type, a target, and an execution timestamp. For example, an action to block IP address 1.2.3.4 will be parsed into an intervention event containing `action_type:block_ip`, `target:1.2.3.4`, and `timestamp:execution_time`.

[0055] Upon receiving the intervention event, this method immediately initiates a defense strategy perturbation process to modify the current Event Causal Evolution Graph (ECEG). Specifically, it simulates the blocking effect of this defense action in a real network environment by changing the graph's topology or edge attributes. The specific modification method depends on the type of intervention event. If the intervention event is a network-level access control event, such as blocking an IP address or port, the method of this invention can identify the entity corresponding to that IP address or port. Subsequently, in the subsequent adversarial intent projection simulation, the conditional transition probability of all potential causal paths involving the controlled entity as a communication source or destination will be dynamically set to zero. Therefore, when extrapolating future attack scenarios, this invention can determine that any attack steps requiring the blocked IP address or port are unachievable.

[0056] If the intervention event is host-level isolation, such as disconnecting an infected host from the network, the method locates the entity representing that host in the ECEG. It then modifies the properties of the edges associated with that host entity. Specifically, the transition probability of all potential future paths originating from that host node and representing outward network communication is set to zero. Simultaneously, the probability of all future network communication paths pointing to that host is also set to zero, thus fully simulating the host's network isolation state in the graph model.

[0057] If the intervention event is an application-level fix, such as installing a specific security patch for an application or system component, this method first identifies the vulnerability targeted by the patch (e.g., a specific CVE number). Then, the invention queries the Global Causal Model (GCM) to find all causal links that exploit the vulnerability. For future events related to the patched asset, the transition probability of these specific causal links is set to zero, meaning an attacker can no longer advance their attack using the patched vulnerability.

[0058] Therefore, through the above method, external defensive actions are precisely translated into quantitative changes to the ECEG graph structure or its parameters, thereby altering the subsequent risk projection process. After the ECEG is modified, this method uses the perturbed graph as a new starting point to re-trigger the adversarial intent projection process. The new projection is entirely based on the updated environment, thus automatically excluding attack paths that have been cut off by defensive measures and re-exploring the attacker's remaining, most likely strategy shifts. Therefore, through the process of execution, perturbation, and re-projection, this invention not only simulates real offensive and defensive confrontations at the model level but also provides a foundation for subsequent evaluation of defensive strategies.

[0059] In one embodiment, the invention further provides forward-looking decision support capabilities, namely, pre-evaluating the potential effectiveness of a set of alternative defense strategies before actually taking any action.

[0060] Specifically, by actively responding to a set of alternative defensive actions A virtual simulation is conducted. And for each potential defensive action in this set of candidates... This invention may perform a separate what-if analysis assuming the action has been performed. Specifically, this invention does not permanently modify the current event causal evolution graph (ECEG), but rather stores the action in memory. A temporary, perturbed copy of ECEG or a computational context is created. Within this temporary context, the invention is based on... The type and target are determined entirely by the logic of the aforementioned defense strategy perturbation, virtually modifying the structure or edge properties of the graph. For example, if If the host is isolated, then in this temporary copy, the transfer probability of all future network communication paths related to host H will be temporarily set to zero.

[0061] And for each potential action After creating the corresponding virtual perturbation graph, this invention will also use this perturbation graph as a basis to target specific attack chains. This involves performing a complete adversarial intent projection and dynamic risk calculation. The result of this calculation assumes that an action has been taken. Then, the attack chain The expected future risks are denoted as In order to quantify the actions To characterize the expected effect, this method introduces an expected risk reduction value, which is calculated as follows: ; In the formula, Indicates the attack chain Take potential defensive actions The expected reduction in risk that can be achieved; Before any new actions are taken, at the current moment attack chain The dynamic risk score is based on the baseline risk calculated from the original, undisturbed ECEG. It is a hypothetical action Under the conditions already implemented, the future risk score is calculated by extrapolating from the virtual disturbance map.

[0062] The essence of this formula is to calculate the difference in risk scores before and after taking action. A larger positive... The value can intuitively indicate the defensive action. It is expected to effectively block or delay the attack process, thereby reducing potential future risks.

[0063] In the alternative set All potential defensive actions All of the above-mentioned virtual simulations and After the value calculation, this method obtains a quantitative evaluation of the effect of each alternative action. At this point, the calculated values ​​can be used to... The value is used to sort all alternative defensive actions in descending order.

[0064] Finally, this sorted list is presented to the security operations personnel. The action at the very top of the list is... The action with the highest value is identified as the optimal or highest priority recommended defense strategy under the current circumstances. Therefore, this method transforms the abstract decision-making process into a quantifiable comparison process based on data and model deduction, providing clear decision support for security operations teams when faced with multiple response options and helping them allocate limited resources to the most effective key points for containing threats.

Claims

1. A big data risk early warning and assessment method based on artificial intelligence, characterized in that, Includes the following steps: Step S1: Collect raw security data from heterogeneous data sources and convert it into a standardized atomic event stream; The atomic event stream includes a unique identifier for each event, a timestamp of the event occurrence, a standardized event type, a source entity, a target entity, and a payload feature vector. Step S2: Construct a global causal model; use transition entropy to quantify the asymmetric information flow between different event types, thereby determining the causal links and associated conditional transition probabilities and typical average time delays; Step S3: Construct and evolve the event causal evolution graph; Based on the real-time received atomic event stream, combined with the basic confidence obtained from the global causal model and the calculated contextual evidence score, use the evidence fusion mechanism to determine the causal confidence, thereby establishing weighted directed edges between nodes; Step S4: Dynamically assess the attacker's capability level; identify the attack chain in the event causal evolution graph, and adjust the attacker's capability level corresponding to the attack chain in real time using dynamic capability update logic based on the execution cost of the attack behavior during the attack chain expansion process. Step S5: Perform adversarial intent projection; starting from the latest node of the attack chain, generate a set of future attack paths through Monte Carlo simulation, wherein the attacker's capability level evaluated in step S4 is used as the dynamic threshold for path pruning, and the conditional probability and expected arrival time of evolving to the terminal intent state are calculated. Step S6: Calculate the dynamic risk score; mathematically fuse the conditional probability, the expected arrival time, and the service impact value of the terminal intent state to calculate the dynamic risk score of the attack chain, and generate an early warning when the score exceeds the early warning threshold.

2. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, In step S2, the step of quantifying the asymmetric information flow between different event types using transfer entropy specifically includes: Extract discrete time series of two event types arranged in chronological order; Calculate the transition entropy from one event type to another, where the transition entropy measures the amount of additional information provided by the historical information of the source event type for predicting the future state of the target event type, given the historical information of the target event type. When the calculated transition entropy is greater than zero and exceeds the predefined causal confidence threshold, the existence of a causal link is confirmed.

3. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, In step S3, determining the causal confidence level using the evidence fusion mechanism specifically includes: Calculate the contextual evidence score, which integrates two dimensions of evidence: entity relevance and temporal pattern similarity. Temporal pattern similarity measures the degree to which the actual time interval is close to the typical average time delay. Evidence fusion is performed by nonlinearly combining the basic confidence score extracted based on the global causal model with the contextual evidence score to generate the final causal confidence score. When the causal confidence level is greater than the confirmation threshold, a directed edge is created between the nodes, and the execution cost queried from the attack cost model is used as the attribute of the edge.

4. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, In step S4, the step of adjusting the attacker's capability level corresponding to the attack chain in real time using dynamic capability update logic specifically includes: Set an initial attacker capability level for newly identified attack chains; When the attack chain completes a new attack action, the execution cost of the new attack action is compared with the current attacker's capability level. If the cost of executing a new action exceeds the current attacker capability level, the excess cost will be proportionally added to the current level based on a preset learning rate factor to obtain an updated attacker capability level. If the cost of implementing the new action does not exceed the current assessment, the attacker's capability level will remain unchanged.

5. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, In step S5, using the attacker's capability level evaluated in step S4 as the dynamic threshold for path pruning specifically includes: During the simulation of adversarial intent projection, the cost of all virtual attack actions on the simulation path is accumulated in real time; Once the cumulative cost of a simulated path exceeds the attacker's capability level corresponding to its source attack chain, the path is determined to be beyond the attacker's capability. Immediately terminate further exploration of the simulated path and implement path pruning.

6. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, In step S6, the calculation of the dynamic risk score of the attack chain is specifically implemented through a dynamic risk calculation function: The function sums up all possible terminal intention states by multiplying the conditional probability by the business impact value and dividing by the result of the time penalty function calculated based on the expected arrival time. The time penalty function is used to amplify the impact of time urgency on risk score, so that the shorter the expected arrival time, the higher the dynamic risk score.

7. The big data risk early warning and assessment method based on artificial intelligence according to claim 6, characterized in that, The method also includes a decision support step based on the expected risk reduction value: For a set of alternative defensive actions, a temporary, perturbed copy of the event causal evolution graph is created in memory for each action. The corresponding alternative defensive actions are virtually executed in the replica, and the adversarial intent projection and dynamic risk score calculation are re-executed to obtain the future risk score; Calculate the expected risk reduction value, which is the difference between the current dynamic risk score and the future risk score; All alternative defensive actions are ranked and recommended based on the expected risk reduction value.

8. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, In step S5, the adversarial intent projection also integrates defensive perturbation constraints, specifically including: Identify the entities or pathways affected by the interventions that have been implemented; During the path expansion process in Monte Carlo simulation, the conditional transition probability of all potential causal paths related to the blocked entity is treated as zero in order to automatically avoid attack paths that have been cut off by defense measures.

9. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, The method further includes a closed-loop feedback step, specifically including: Receive intervention events from external input, wherein the intervention events include action type, target, and execution timestamp; Modify the topology or edge properties of the event causal evolution graph based on the intervention event; Based on the modified graph structure, steps S5 and S6 are re-executed to achieve dynamic redirection of causal inference.

10. The big data risk early warning and assessment method based on artificial intelligence according to claim 1, characterized in that, The method also includes: pre-constructing an attack cost model and a terminal intent state library; The attack cost model is used to assign a quantified execution cost to each standardized attack action; The terminal intent state library is used to define terminal intent states and assign quantified service impact values ​​to each state.

Images