BAS safety effectiveness verification method, system, device and medium based on environmental perception and multi-modal simulation

Abstract

This invention provides a method, system, device, and medium for verifying the security effectiveness of a BAS (Blockchain Assurance System) based on environmental awareness and multimodal simulation, relating to the field of network security technology. The method includes: dynamically perceiving the target network's environment, collecting static and dynamic data from the target network, and constructing a knowledge graph; modeling the attack process as a Markov decision process, and using the knowledge graph and deep reinforcement learning algorithms to solve for the optimal attack strategy to achieve the preset attack target under the current environmental state; generating an attack chain based on the optimal attack strategy, and simulating the execution of the attack chain in a real environment by selecting one of several preset simulation modalities, and dynamically adjusting the optimal attack strategy based on real-time feedback of the execution results; and quantitatively evaluating the system's security effectiveness based on the expected cumulative reward under the optimal attack strategy. This invention significantly improves the accuracy and realism of security effectiveness evaluation in intrusion and attack simulation systems.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to a method, system, device, and medium for verifying the security effectiveness of BAS based on environmental awareness and multimodal simulation. Background Technology

[0002] With the rapid development of network technology, cybersecurity threats are becoming increasingly severe. For enterprises, comprehensively reviewing the overall security status of the organization is a challenging task. Traditional testing methods, such as security testing and blue team services, have limitations. To ensure the continued effectiveness of security mechanisms, more comprehensive security capabilities, and faster and broader threat defense coverage, a more routine security verification strategy is needed. From both offensive and defensive perspectives, introducing BAS (Breach and Attack Simulation) has become an effective method for enterprises to continuously verify their defense capabilities. By simulating attack methods, BAS can help enterprises better prevent unknown risks and comprehensively enhance their security operations capabilities.

[0003] The areas for improvement in the current mainstream BAS technology are:

[0004] (1) Lack of environmental context awareness: When testing existing BAS systems, deep environmental information such as the real-time status of the target system, network topology, and business logic dependencies is often ignored. This makes the testing "blind", which may trigger a large number of alarms that are not related to actual business, or fail to detect deep vulnerabilities existing in complex business interactions; (2) Rigid attack chains with poor adaptability: Predefined, linear attack chains cannot simulate the dynamic decision-making process of real hackers. Real attackers adjust their strategies in real time based on the results of the previous attack. The static attack chains of existing technologies cannot achieve this kind of "roundabout" and "adaptive" penetration, resulting in insufficient assessment of the effectiveness of the defense-in-depth system; (3) The assessment results are one-sided and have low accuracy: Because the tests are atomic and static, the assessment results can only reflect the vulnerability of "a certain point" and cannot quantify the overall security status of the entire "attack surface". Security teams find it difficult to accurately determine the actual risk level and the most likely path to be breached from results such as "50 out of 1,000 tests fail". The accuracy is low and the guidance is limited. (4) Inability to effectively simulate advanced persistent threats (APTs): The core of APT attacks lies in long-term, slow, and multi-stage penetration. The existing BAS's short-cycle, high-noise testing mode is difficult to effectively simulate such attacks and cannot verify the enterprise's true resistance to APTs. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method, system, device, and medium for verifying the security effectiveness of BAS based on environmental perception and multimodal simulation, so as to significantly improve the accuracy and realism of security effectiveness assessment of intrusion and attack simulation systems through dynamic environmental perception, attack graph and Markov decision process (MDP) fusion, and multimodal attack chain simulation.

[0006] This application provides the following technical solution: a BAS security effectiveness verification method based on environmental perception and multimodal simulation, comprising: Dynamic environmental perception is performed on the target network, static and dynamic data in the target network are collected, and a knowledge graph representing the network space environment is constructed based on the collected data; The attack process is modeled as a Markov decision process. Based on the knowledge graph, the optimal attack strategy to achieve the preset attack target under the current environmental state is solved by deep reinforcement learning algorithm. An attack chain is generated based on the optimal attack strategy. In a real environment, one of the preset simulation modes is selected to simulate and execute the attack chain. The optimal attack strategy is dynamically adjusted based on the real-time feedback of the execution results. The system's security effectiveness is quantitatively evaluated based on the expected cumulative reward under the optimal attack strategy.

[0007] According to one embodiment of this application, static and dynamic data in a target network are collected, and a knowledge graph representing the cyberspace environment is constructed based on the collected data, including: Automatically collect asset data, vulnerability data, identity and permission data, network topology and access control policies, and business logic flow data of the target network through one or more methods such as lightweight proxy, network scanning, and API interface; A knowledge graph G is constructed based on the collected data, using an attribute graph model. G = (V, E, A_V, A_E); Where V is the set of nodes, representing assets; E is the set of edges, representing the relationships between nodes; A_V is the node attribute; and A_E is the edge attribute.

[0008] According to one embodiment of this application, the attack process is modeled as a Markov decision process, including: formally defining the attack process as a Markov decision process tuple (S,A,P,R,γ): Where S is the state space, which is composed of feature vectors encoding the attacker's posture in the knowledge graph; A is the action space, where each action corresponds to a relation edge or an attack technique in the knowledge graph; P is the state transition rate, which represents the probability of transitioning to a new state s_{t+1} after performing action a_t in state s_t; R is the reward function, which is used to guide the attacker to achieve the attack goal; and γ is the discount factor.

[0009] According to one embodiment of this application, in the Markov decision process tuple, the reward function R is defined as: R(s_t,a_t,s_{t+1})=w_1 I(s_{t+1})+w_2 C(a_t)+w_3 ΔP(s_t,s_{t+1}) Where R(s_t,a_t,s_{t+1}) represents the reward function oriented towards achieving the attack goal, I(s_{t+1}) represents the intelligence value reward, C(a_t) represents the concealment cost penalty, ΔP(s_t,s_{t+1}) represents the privilege enhancement gain reward, indicating the degree of privilege enhancement obtained by the attacker before and after the state transition, and w_1, w_2, and w_3 represent the corresponding weight coefficients.

[0010] According to one embodiment of this application, the optimal attack strategy for achieving a preset attack target under the current environmental state is solved using a deep reinforcement learning algorithm, and further includes: The Markov decision process is solved using a deep Q-network algorithm. The optimal attack strategy from state to action is learned by offline training of the deep Q-network in a simulation environment. The offline training phase of the deep Q-network includes: Initialize the master Q-network and the target Q-network, and initialize the experience playback buffer; In each training round, actions are selected and executed according to the ε-greedy policy. The immediate reward and the next state obtained after executing the action are observed. The experience tuple consisting of the current state, the executed action, the reward obtained, and the next state is stored in the experience replay buffer. A batch of experience tuples is randomly sampled from the experience replay buffer, a target value is calculated using the target Q network, and the parameters of the main Q network are updated by minimizing the loss function based on the target value. Every preset number of steps, the parameters of the main Q network are copied to the target Q network.

[0011] According to one embodiment of this application, an attack chain is simulated and executed by selecting one of a plurality of preset simulation modes, and the optimal attack strategy is dynamically adjusted based on real-time feedback of the execution results, including: For each attack action in the attack chain, based on the attack action type and the current simulation environment, the optimal simulation mode is determined from a set of multiple simulation modes for simulation execution; the multiple simulation modes include: real load mode, simulated traffic mode, and logic verification mode; During the execution of the attack chain, environmental feedback is monitored in real time. If the actual state transition does not match the expectation, the optimal attack strategy is re-invoked based on the new state, the optimal attack action is calculated, and the subsequent attack chain is dynamically adjusted.

[0012] According to one embodiment of this application, a quantitative evaluation of the system security effectiveness is performed based on the expected cumulative reward under the optimal attack strategy, including: The security state value of the entire system is defined as the reciprocal of the minimum expected cost to reach the preset critical target state from the initial attack state, and the security state value of the system is calculated. Based on the frequency of each asset node and access relationship edge appearing and their corresponding contribution in the attack chain generated by the optimal attack strategy, the nodes and edges in the knowledge graph are colored and rendered to generate an attack surface heatmap for visualization.

[0013] This application also provides a BAS safety effectiveness verification system based on environmental perception and multimodal simulation, including: The environmental perception and graph construction module is used to perform dynamic environmental perception on the target network, collect static and dynamic data in the target network, and construct a knowledge graph representing the network spatial environment based on the collected data. The attack path planning module is used to model the attack process as a Markov decision process, and based on the knowledge graph, solves the optimal attack strategy to achieve the preset attack target under the current environment state through deep reinforcement learning algorithm. The multimodal simulation module is used to generate an attack chain based on the optimal attack strategy, and in a real environment, select one of the preset simulation modes to simulate and execute the attack chain, and dynamically adjust the optimal attack strategy based on the real-time feedback of the execution results. The evaluation module is used to quantitatively evaluate the security effectiveness of the system based on the expected cumulative reward under the optimal attack strategy.

[0014] This application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described BAS security effectiveness verification method based on environmental perception and multimodal simulation.

[0015] This application also provides a computer-readable storage medium storing a computer program that performs the above-described BAS security effectiveness verification method based on environmental awareness and multimodal simulation.

[0016] Compared with the prior art, the beneficial effects achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: the embodiments of this invention, through dynamically constructed knowledge graphs, enable BAS testing to closely fit the real business environment, avoiding invalid and disconnected testing, and significantly improving the accuracy of evaluation; by simulating the adaptive decision-making process of attackers through Markov Decision Process (MDP) and reinforcement learning, complex and circuitous attack paths that cannot be identified by static analysis can be discovered, greatly enhancing the ability to simulate advanced threats; the introduction of quantitative security indicators based on probability and expected reward makes the security status measurable and comparable, providing clear data support and optimization direction for security investment; and the intelligent selection of execution modes according to the risks of different test actions, minimizing interference and risks to the production system while ensuring the evaluation effect. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a BAS safety effectiveness verification method based on environmental perception and multimodal simulation, provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the process of solving the optimal attack strategy in a BAS security effectiveness verification method based on environmental perception and multimodal simulation, as provided in an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the composition of a BAS safety effectiveness verification system based on environmental perception and multimodal simulation, as provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] like Figure 1 As shown, this embodiment of the invention provides a BAS security effectiveness verification method based on environmental perception and multimodal simulation. This method can dynamically perceive the environment, intelligently generate and execute multimodal attack chains, and quantify and output security effectiveness indicators, thereby significantly improving the accuracy and realism of security assessments. The method includes: S101. Perform dynamic environment perception on the target network, collect static and dynamic data in the target network, and construct a knowledge graph representing the network space environment based on the collected data; S102. The attack process is modeled as a Markov decision process. Based on the knowledge graph, the optimal attack strategy to achieve the preset attack target under the current environment state is solved by deep reinforcement learning algorithm. S103. Generate an attack chain according to the optimal attack strategy, and in a real environment, select one of the preset simulation modes to simulate and execute the attack chain, and dynamically adjust the optimal attack strategy based on the real-time feedback of the execution results. S104. Based on the expected cumulative reward under the optimal attack strategy, quantitatively evaluate the security effectiveness of the system.

[0022] In specific implementation, step S101 includes the following: 1. Data Acquisition: Automated acquisition of asset data, vulnerability data, identity and permission data, network topology and access control policies, and business logic flow data of the target network through one or more methods, including lightweight proxies, network scanning, and API interfaces. Asset data includes a list and configuration of hosts, servers, network devices, and applications; vulnerability data consists of CVE information obtained through vulnerability scanners; identity and permission data includes users, user groups, service accounts, and permission assignments (such as Active Directory domain information); network topology and access control policies include firewall rules, routing tables, VLAN segmentation, etc.; and business logic flow consists of API calls and data flows between critical business applications.

[0023] 2. Knowledge Graph Construction: Based on the collected data, a knowledge graph G of attribute graph model is constructed: G = (V, E, A_V, A_E); Where V is the set of nodes, representing assets such as hosts, users, and applications; E is the set of edges, representing relationships between nodes such as accessibility, ownership, or vulnerability; A_V are node attributes such as host operating system and user permission level; and A_E are edge attributes such as access protocol and vulnerability exploitation success rate estimate.

[0024] Step S102 specifically includes the following: 1. Problem Modeling: The attack process is modeled as a Markov decision process tuple (S, A, P, R, γ): where S is the state space, consisting of feature vectors encoding the attacker's posture in the knowledge graph; A is the action space, where each action corresponds to a relation edge or an attack technique in the knowledge graph; P is the state transition rate, representing the probability of transitioning to a new state s_{t+1} after performing action a_t in state s_t; R is the reward function, used to guide the attacker to achieve the attack goal; and γ is the discount factor.

[0025] Specifically, S is the state-environment encoding of the attacker's posture, representing the attacker's state at a certain moment. It is a feature vector representing the state s_t=[L_t,P_t,C_t,D_t], which encodes the attacker's current privilege level, network location, and set of controlled hosts, etc., and is expressed as: s_t∈S, which uniquely describes the entire simulation environment (network) and the posture achieved by the attacker at time t.

[0026] Where L_t(Location&Access): is a network location and access permission vector, representing the set of network segments and hosts that the attacker currently controls or can access, for example, [Internet_DMZ, Internal_Web, Internal_DB, Domain_Controller]. If the attacker is in the DMZ zone, the corresponding position is 1; P_t (Privilege Level): Privilege level vector, representing the level of privileges an attacker has on assets they control. For example, each type of asset can be associated with a privilege value: 0: no privileges; 1: user privileges; 2: administrator privileges; 3: SYSTEM / root privileges; P_t is a set of these values ​​or a comprehensive score. C_t (Compromised Assets): The criticality of the compromised assets, indicating the value or criticality of the assets that have been controlled (e.g., ordinary PC = 1, file server = 5, domain controller = 10), representing the value that the attacker has obtained; D_t (Detection Alert Level): Detection Alert Level, an indicator of whether the current attack has triggered an alert from security devices (such as IDS, firewalls, EDR). 0 indicates complete concealment, 1 indicates a low-level alert, and 2 indicates a high-level alert. This directly affects the choice of action.

[0027] A represents an action, indicating an operation that an attacker can perform. Each action corresponds to an edge in the knowledge graph G or a specific attack technique; it is represented as: a_t∈A represents the action performed at time t. Actions are directly mapped to edges in the knowledge graph, and each action a must be associated with nodes and edges in the knowledge graph G, as detailed in Table 1 below: Table 1. Relationship between Actions and Knowledge Graphs

[0028] P represents the state transition rate. P(s_{t+1}|s_t,a_t) is a probability distribution representing the probability of transitioning to a new state s_{t+1} after performing action a_t in state s_t. This probability can be learned from the edge attributes of the knowledge graph (such as vulnerability exploitation success rate) and system monitoring data (such as service availability). The initial P is estimated based on static knowledge, including: vulnerability exploitation success rate: based on CVSS score, vulnerability exploitation maturity, patch status, etc.; service availability: for unknown situations, a conservative default probability, such as 0.5, can be set.

[0029] R is the reward function, representing the immediate reward after performing the action. We design a reward function oriented towards achieving the attack objective, guiding the intelligent direction of the attack simulation; this function is defined as follows: R(s_t,a_t,s_{t+1})=w_1 I(s_{t+1})+w_2 C(a_t)+w_3 ΔP(s_t,s_{t+1}); Where I(s_{t+1}) is the intelligence value, a high reward is given if the new state s_{t+1} approaches or achieves the preset attack target, this is a sparse reward; C(a_t) is the concealment cost, the greater the noise of action a_t, the higher the risk of being detected, the lower the reward or the higher the penalty; C(a_t)∝ -log(1 - DetectionRate(a_t)); ΔP(s_t,s_{t+1}) is the privilege enhancement gain, representing the degree of privilege enhancement obtained from s_t to s_{t+1}; w_1, w_2, w_3 are weight coefficients used to balance the importance of different targets.

[0030] γ is a discount factor, γ∈[0,1], used to measure the current value of future rewards.

[0031] 2. Strategy Solving: Find a target policy π(s) that specifies which action a should be taken in each state s to maximize the expected cumulative discount reward or achieve the attack objective with the highest cost-effectiveness (i.e., the state value function V^π(s)): Using Markov decision process tuples, initial state s_θ, and target state set S_target as input, a deep Q-network is employed to solve for the optimal attack strategy in the current environment. The core algorithm and workflow include: Since the state space can be very large (high-dimensional and continuous), a deep reinforcement learning algorithm, Deep Q-Network (DQN), is used to solve it. The process consists of two stages: offline training and online application, as follows: Figure 2 As shown.

[0032] Phase 1: Offline Training (Model learning, not involving system production operations; training is conducted in a dedicated simulation environment during this phase) (1) Network initialization Initialize two neural networks with identical structures: Main-Net: Parameters are θ. Input is state s, output is the Q-value vector corresponding to all actions a∈A; Target-Net: The parameter is θ^-. Initially, θ^- = 0; Initialize the experience replay buffer D with a capacity of N (e.g., 1,000,000 experiences).

[0033] (2) Interaction and experience collection FOR episode=1 TO M (Total training rounds) DO: Reset the environment to obtain the initial state s_θ; FOR t=1 TO T (maximum number of steps per round) DO: Randomly select an action (exploration) with probability ε (exploration rate, initially 1.0, gradually decaying); select argmax_{a} Q(s_t, a; θ) (exploitation) with probability 1-ε. Execute action a_t in a simulation environment and observe the resulting reward r_t and the new state s_{t+1}. Here, "execution" takes place in a highly simulated sandbox or test environment to compute P(s_{t+1}|s_t, a_t) and R(s_t,a_t, s_{t+1}).

[0034] Store the experience tuple (s_t, a_t, r_t, s_{t+1}) into the experience replay buffer D; where s_t represents the current state, a_t represents the action performed, r_t represents the reward obtained, and s_{t+1} represents the next state (i.e., the new state).

[0035] (3) Network training (learning from experience) If the amount of data in the buffer is greater than the size of a batch (e.g., 32), then: Randomly sample a batch (e.g., 32) of empirical tuples (s_i, a_i, r_i, s_{i+1}) from buffer D. (Random sampling can break the correlation between data, making training more stable.) If s_{i+1} is in a terminating state (goal achieved or attack failed), then the target value y_i = r_i; otherwise, the target value y_i = r_i + γ max_{a} Q(s_{i+1},a; θ^-). (Utilize the target network to calculate the expected future return and reduce training oscillations) Calculate the loss function. Use the mean squared error (MSE) to measure the difference between the Q-network's predicted value and the target value: L(θ) = 1 / NΣ[y_i - Q(s_i,a_i;θ)] 2 ; Use gradient descent algorithms (such as Adam) to minimize the loss function L(θ) and update the parameters θ of the Q network (Main-Net); Every C steps, the parameters θ of the Q network are copied to the target Q network: θ^- ← θ.

[0036] (4) Exploration rate decay As training progresses, the exploration rate ε is gradually reduced from a high value (e.g., 1.0) to a low value (e.g., 0.01), allowing the agent to gradually transition from random exploration to utilizing learned superior strategies.

[0037] Phase Two: Online Application (Policy Execution). After the model training is complete, the optimal Q-network (i.e., policy π) will be frozen. Used for safety assessment of real production systems: (1) Strategy loading The trained Q-network Q(s,a;θ) Loaded into the BAS engine. At this point, policy π (s) = argmax_{a} Q(s,a;θ) The result is deterministic and no longer requires random exploration.

[0038] (2) Real-time path planning The BAS system starts from the initial state s_0. In each state s_t, the BAS engine queries π. (s_t) is used to obtain the current optimal attack action a_t. Then, the system switches to multimodal attack chain simulation and executes the action using multimodal simulation. Based on the execution result, the environment enters a new state s_{t+1}. This process is repeated until the target state S_target is reached or the maximum number of steps is exceeded. The final result is an optimal attack strategy for the current environment.

[0039] Step S103 specifically includes the following: (1) Attack chain generation: Starting from the initial state s_0, the attack chain is generated according to the optimal policy π obtained by solving the algorithm. Generate the attack chain with the highest probability: chain={a_0,a_1,a_2……,a_n}, where each action a_i is generated by π. (s_i) determines.

[0040] (2) Multimodal simulation execution: The BAS engine does not blindly execute the chain (attack chain), but selects the optimal simulation mode to execute based on the action type and the current environment; Realistic payload mode: Use realistic or harmless payloads for non-destructive, high-success-rate actions (such as information gathering); Simulated traffic mode: For high-risk actions (such as vulnerability exploitation), simulated traffic with the same network traffic characteristics as real attacks but stripped of harmful payloads is generated. It is only used to trigger IDS / IPS and verify its effectiveness, so as to avoid affecting business. Logical verification mode: For actions such as privilege escalation, the feasibility is logically verified by querying the system API or configuration, rather than actually executing the action.

[0041] (3) Adaptive adjustment: During execution, environmental feedback is monitored in real time. If an action a_i fails (state transition does not occur as expected), the current state s_i and failure information are immediately fed back to the MDP model, and Q is recalculated. and π This allows for dynamic bypassing of failed paths and selection of the latest optimal action, i.e.: chain={…,a_ii,a_jj,…}, where a_ii is the optimal action based on the new state s_i.

[0042] Step S104 specifically includes the following: (1) Calculate the system security state value: Define the security state value of the entire system as the reciprocal of the minimum expected cost to reach the critical target state s_target (such as domain control privileges) from the initial attack state s_0; Security_State_Value = 1 / V^π (s_0); Where V^π (s_0) represents the expected cumulative reward for reaching s_target from s_0 by following the optimal strategy. (Since the reward function includes costs, a higher cumulative reward indicates a lower attack cost and a less secure system.) This value comprehensively reflects the attack difficulty and path diversity.

[0043] (2) Calculate the exposure probability of critical attack paths: For the optimal attack strategy π The first K optimal attack paths {chain_1, chain_2, ..., chain_k} are generated, and the exposure probability (i.e. the probability of being successfully executed) of each path is calculated.

[0044] (3) Generate a heatmap report: Based on the exposure probability and the frequency of occurrence of each asset node and access relationship edge in the critical attack path, color and render the nodes and edges in the knowledge graph. Specifically, the nodes and edges in the knowledge graph G are colored according to their frequency of occurrence and contribution in the critical attack path to generate an attack surface heatmap, which intuitively displays the most vulnerable assets and access relationships.

[0045] By employing embodiments of this invention, a dynamically constructed knowledge graph enables BAS testing to closely align with the real business environment, avoiding ineffective and disconnected testing and significantly improving the accuracy of the assessment. Through Markov Decision Process (MDP) and reinforcement learning simulation of the attacker's adaptive decision-making process, complex and circuitous attack paths that static analysis cannot identify can be discovered, greatly enhancing the ability to simulate advanced threats. The introduction of quantitative security indicators based on probability and expected reward makes the security status measurable and comparable, providing clear data support and optimization directions for security investments. Based on the risk of different testing actions, the execution mode is intelligently selected, ensuring the assessment effect while minimizing interference and risk to the production system.

[0046] like Figure 3 As shown, this application also provides a BAS safety effectiveness verification system 200 based on environmental perception and multimodal simulation, comprising: The environment perception and graph construction module 201 is used to perform dynamic environment perception on the target network, collect static and dynamic data in the target network, and construct a knowledge graph representing the network spatial environment based on the collected data. The attack path planning module 202 is used to model the attack process as a Markov decision process, and based on the knowledge graph, solve the optimal attack strategy to achieve the preset attack target under the current environment state through a deep reinforcement learning algorithm. The multimodal simulation module 203 is used to generate an attack chain according to the optimal attack strategy, and select one of the preset simulation modes to simulate and execute the attack chain in a real environment, and dynamically adjust the optimal attack strategy based on the real-time feedback of the execution results. Evaluation module 204 is used to quantitatively evaluate the security effectiveness of the system based on the expected cumulative reward under the optimal attack strategy.

[0047] In specific implementation, the environment perception and graph construction module 201 automatically collects asset data, vulnerability data, identity and permission data, network topology and access control policies, and business logic flow data of the target network through one or more methods such as lightweight proxy, network scanning, and API interface; and constructs a knowledge graph G based on the collected data.

[0048] In specific implementation, the attack path planning module 202 formally defines the attack process as a Markov decision process tuple (S, A, P, R, γ). A deep Q-network algorithm is used to solve the Markov decision process. By training the deep Q-network offline in a simulation environment, the optimal attack strategy from state to action is learned. The offline training phase of the deep Q-network includes: initializing the main Q-network and the target Q-network, and initializing the experience replay buffer; in each training round, selecting and executing actions according to an ε-greedy policy, observing the immediate reward and the next state obtained after executing the action, and storing the experience tuple consisting of the current state, the executed action, the obtained reward, and the next state into the experience replay buffer; randomly sampling a batch of experience tuples from the experience replay buffer, calculating the target value using the target Q-network, and updating the parameters of the main Q-network based on the target value by minimizing the loss function; and copying the parameters of the main Q-network to the target Q-network every preset number of steps.

[0049] In specific implementation, the multimodal simulation module 203 is specifically used to: for each attack action on the attack chain, determine the optimal simulation mode from a set of preset simulation modes for simulation execution based on the attack action type and the current simulation environment; the set of simulation modes includes: real load mode, simulated traffic mode, and logic verification mode; during the execution of the attack chain, monitor the environmental feedback in real time, and if the actual state transition does not match the expectation, re-invoke the optimal attack strategy based on the new state, calculate the optimal attack action, and dynamically adjust the subsequent attack chain.

[0050] In specific implementation, the evaluation module 204 is specifically used to: define the security state value of the entire system as the reciprocal of the minimum expected cost to reach the preset key target state from the initial attack state, and calculate the security state value of the system; based on the frequency of occurrence and corresponding contribution of each asset node and access relationship edge in the attack chain generated by the optimal attack strategy, color and render the nodes and edges in the knowledge graph to generate an attack surface heatmap for visualization.

[0051] In one embodiment, a computer device is provided, such as Figure 4 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory 301 and capable of running on the processor 302. The processor 302 executes the BAS security effectiveness verification method based on environmental perception and multimodal simulation.

[0052] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0053] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that performs the above-described BAS security effectiveness verification method based on environmental awareness and multimodal simulation.

[0054] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media does not include transient media, such as modulated data signals and carrier waves.

[0055] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0056] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for verifying the safety effectiveness of a BAS (Balanced Assurance System) based on environmental perception and multimodal simulation, characterized in that, include: Dynamic environmental perception is performed on the target network, static and dynamic data in the target network are collected, and a knowledge graph representing the network space environment is constructed based on the collected data; The attack process is modeled as a Markov decision process. Based on the knowledge graph, the optimal attack strategy to achieve the preset attack target under the current environmental state is solved by deep reinforcement learning algorithm. An attack chain is generated based on the optimal attack strategy. In a real environment, one of the preset simulation modes is selected to simulate and execute the attack chain. The optimal attack strategy is dynamically adjusted based on the real-time feedback of the execution results. The system's security effectiveness is quantitatively evaluated based on the expected cumulative reward under the optimal attack strategy.

2. The method according to claim 1, characterized in that, Collect static and dynamic data from the target network, and construct a knowledge graph representing the cyberspace environment based on the collected data, including: Automatically collect asset data, vulnerability data, identity and permission data, network topology and access control policies, and business logic flow data of the target network through one or more methods such as lightweight proxy, network scanning, and API interface; A knowledge graph G is constructed based on the collected data, using an attribute graph model. G = (V, E, A_V, A_E); Where V is the set of nodes, representing assets; E is the set of edges, representing the relationships between nodes; A_V is the node attribute; and A_E is the edge attribute.

3. The method according to claim 1, characterized in that, Modeling the attack process as a Markov decision process includes: formally defining the attack process as a Markov decision process tuple (S,A,P,R,γ): Where S is the state space, which is composed of feature vectors encoding the attacker's posture in the knowledge graph; A is the action space, where each action corresponds to a relation edge or an attack technique in the knowledge graph; P is the state transition rate, which represents the probability of transitioning to a new state s_{t+1} after performing action a_t in state s_t; R is the reward function, which is used to guide the attacker to achieve the attack goal; and γ is the discount factor.

4. The method according to claim 3, characterized in that, In the Markov decision process tuple, the reward function R is defined as: R(s_t,a_t,s_{t+1})=w_1 I(s_{t+1})+w_2 C(a_t)+w_3 ΔP(s_t,s_{t+1}) Where R(s_t,a_t,s_{t+1}) represents the reward function oriented towards achieving the attack goal, I(s_{t+1}) represents the intelligence value reward, C(a_t) represents the concealment cost penalty, ΔP(s_t,s_{t+1}) represents the privilege enhancement gain reward, indicating the degree of privilege enhancement obtained by the attacker before and after the state transition, and w_1, w_2, and w_3 represent the corresponding weight coefficients.

5. The method according to claim 1, characterized in that, The method of using deep reinforcement learning algorithms to solve for the optimal attack strategy to achieve the preset attack target under the current environment also includes: The Markov decision process is solved using a deep Q-network algorithm. The optimal attack strategy from state to action is learned by offline training of the deep Q-network in a simulation environment. The offline training phase of the deep Q-network includes: Initialize the master Q-network and the target Q-network, and initialize the experience playback buffer; In each training round, actions are selected and executed according to the ε-greedy policy. The immediate reward and the next state obtained after executing the action are observed. The experience tuple consisting of the current state, the executed action, the reward obtained, and the next state is stored in the experience replay buffer. A batch of experience tuples is randomly sampled from the experience replay buffer, a target value is calculated using the target Q network, and the parameters of the main Q network are updated by minimizing the loss function based on the target value. Every preset number of steps, the parameters of the main Q network are copied to the target Q network.

6. The method according to claim 1, characterized in that, Selecting one of a variety of preset simulation modes to simulate the attack chain, and dynamically adjusting the optimal attack strategy based on real-time feedback of the execution results, including: For each attack action in the attack chain, based on the attack action type and the current simulation environment, the optimal simulation mode is determined from a set of multiple simulation modes for simulation execution; the multiple simulation modes include: real load mode, simulated traffic mode, and logic verification mode; During the execution of the attack chain, environmental feedback is monitored in real time. If the actual state transition does not match the expectation, the optimal attack strategy is re-invoked based on the new state, the optimal attack action is calculated, and the subsequent attack chain is dynamically adjusted.

7. The method according to claim 1, characterized in that, Based on the expected cumulative reward under the aforementioned optimal attack strategy, the system security effectiveness is quantitatively evaluated, including: The security state value of the entire system is defined as the reciprocal of the minimum expected cost to reach the preset critical target state from the initial attack state, and the security state value of the system is calculated. Based on the frequency of each asset node and access relationship edge appearing and their corresponding contribution in the attack chain generated by the optimal attack strategy, the nodes and edges in the knowledge graph are colored and rendered to generate an attack surface heatmap for visualization.

8. A BAS safety effectiveness verification system based on environmental perception and multimodal simulation, characterized in that, include: The environmental perception and graph construction module is used to perform dynamic environmental perception on the target network, collect static and dynamic data in the target network, and construct a knowledge graph representing the network spatial environment based on the collected data. The attack path planning module is used to model the attack process as a Markov decision process, and based on the knowledge graph, solves the optimal attack strategy to achieve the preset attack target under the current environment state through deep reinforcement learning algorithm. The multimodal simulation module is used to generate an attack chain based on the optimal attack strategy, and in a real environment, select one of the preset simulation modes to simulate and execute the attack chain, and dynamically adjust the optimal attack strategy based on the real-time feedback of the execution results. The evaluation module is used to quantitatively evaluate the security effectiveness of the system based on the expected cumulative reward under the optimal attack strategy.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the BAS security effectiveness verification method based on environmental perception and multimodal simulation as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that performs the BAS security effectiveness verification method based on environmental perception and multimodal simulation as described in any one of claims 1 to 7.

Images