Cyber-attack path dynamic programming method for power information system

Abstract

The application relates to an attack path dynamic programming method of a power information system. The method comprises the following steps: acquiring multi-dimensional state data of a target power information system, and constructing a state vector based on the multi-dimensional state data; determining an attack action sequence based on the state vector, and determining an executable attack path according to the attack action sequence; determining attack action instructions to be issued according to the executable attack path, and issuing the attack action instructions; receiving returned execution feedback information corresponding to the attack action instructions, and state data of the target power information system, and updating the state vector based on the state data; determining a path correction factor based on the state data and the execution feedback information; and when the path correction factor meets a preset re-planning trigger condition, re-planning the executable attack path based on the updated state vector. The scheme can improve the adaptability of the attack path to a dynamic test environment in the process of penetration testing of the power information system.

Description

Technical Field

[0001] This application relates to the field of power network technology, and in particular to a dynamic planning method for attack paths in power information systems. Background Technology

[0002] Power information systems are widely used in scenarios such as dispatching, monitoring, and business management, and their secure operation is directly related to the stability of power services. To assess potential security risks to power information systems, penetration testing is typically used to simulate network attack processes.

[0003] In existing penetration testing of power information systems, attack paths are typically predetermined, and testing is conducted according to these predetermined paths. However, power information systems are characterized by frequent environmental changes, and the applicability of predetermined attack paths may decrease during testing, thus affecting the normal progress of subsequent penetration tests.

[0004] Currently, there is no effective solution to improve the adaptability of attack paths to dynamic testing environments during penetration testing of power information systems. Summary of the Invention

[0005] Therefore, it is necessary to provide a dynamic planning method for attack paths in power information systems that can improve the adaptability of attack paths to dynamic testing environments during penetration testing of power information systems, in order to address the aforementioned technical problems.

[0006] Firstly, this application provides a dynamic planning method for attack paths in a power information system, including:

[0007] Acquire multidimensional state data of the target power information system, and construct the state vector of the target power information system based on the multidimensional state data;

[0008] The attack action sequence is determined based on the state vector, and the executable attack path is determined based on the attack action sequence.

[0009] The attack action command to be issued is determined based on the executable attack path, and the attack action command is issued to the test execution terminal.

[0010] Receive the execution feedback information corresponding to the attack action instruction returned by the test execution terminal, as well as the status data of the target power information system, and update the status vector based on the status data;

[0011] Based on the state data and the corresponding expected state information, as well as the execution feedback information and the corresponding expected execution information, a path correction factor is determined;

[0012] When the path correction factor meets the preset replanning trigger condition, the executable attack path is replanned based on the updated state vector.

[0013] In one embodiment, determining the attack action sequence based on the state vector includes:

[0014] Based on a pre-constructed Markov decision process model, multiple candidate attack actions are determined for the system state represented by the state vector.

[0015] The action value corresponding to each candidate attack action is determined using a pre-trained decision model, and the attack action sequence is determined based on the action value corresponding to each candidate attack action.

[0016] In one embodiment, the Markov decision process model includes a state space, an action space, a state transition probability function, a multi-objective weighted reward function, and a discount factor; wherein, each system state in the state space is represented by a corresponding state vector, the action space includes multiple attack actions pre-set for the target power information system; the state transition probability function is used to determine the probability of transitioning to the next system state after executing the candidate attack action in the corresponding system state, the multi-objective weighted reward function is used to determine the reward value of executing the candidate attack action in the corresponding system state, and the discount factor is used to adjust the weight of the reward value corresponding to the subsequent system state.

[0017] In one embodiment, the state vector includes asset attribute characteristics, network service characteristics, security risk characteristics, access permission characteristics, defense status characteristics, and operational status characteristics.

[0018] In one embodiment, determining the executable attack path based on the attack action sequence includes: generating an initial attack path based on the attack action sequence, and performing constraint verification on the initial attack path based on preset power service constraints to determine the executable attack path.

[0019] In one embodiment, the power service constraints include network performance constraints, service security constraints, permission compliance constraints, and behavior concealment constraints; wherein, the network performance constraints are used to determine whether the initial attack path meets preset network performance requirements, the service security constraints are used to determine whether the service access and testing behaviors involved in the initial attack path meet preset service security requirements, the permission compliance constraints are used to determine whether the attack actions in the initial attack path meet preset permission compliance requirements, and the behavior concealment constraints are used to determine whether the initial attack path meets preset behavior concealment requirements.

[0020] In one embodiment, determining the path correction factor based on the state data and the corresponding expected state information, and the execution feedback information and the corresponding expected execution information, includes:

[0021] The state deviation amount is determined based on the difference between the state data and the corresponding expected state information;

[0022] The execution deviation is determined based on the difference between the execution feedback information and the corresponding expected execution information;

[0023] The path correction factor is determined based on the state deviation and the execution deviation.

[0024] In one embodiment, after receiving the execution feedback information and the state data, the dynamic planning method for attack paths in the power information system further includes: updating the model parameters of the decision model based on the execution feedback information and the state data, so that the updated decision model can be used to determine the subsequent attack action sequence.

[0025] In a second aspect, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0026] The system acquires multidimensional state data of the target power information system and constructs a state vector of the target power information system based on the multidimensional state data; determines an attack action sequence based on the state vector and determines an executable attack path based on the attack action sequence; determines an attack action command to be issued based on the executable attack path and issues the attack action command to the test execution terminal; receives execution feedback information corresponding to the attack action command returned by the test execution terminal, as well as the state data of the target power information system, and updates the state vector based on the state data; determines a path correction factor based on the state data and the corresponding expected state information, and the execution feedback information and the corresponding expected execution information; when the path correction factor meets a preset replanning trigger condition, replans the executable attack path based on the updated state vector.

[0027] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the following steps:

[0028] The system acquires multidimensional state data of the target power information system and constructs a state vector of the target power information system based on the multidimensional state data; determines an attack action sequence based on the state vector and determines an executable attack path based on the attack action sequence; determines an attack action command to be issued based on the executable attack path and issues the attack action command to the test execution terminal; receives execution feedback information corresponding to the attack action command returned by the test execution terminal, as well as the state data of the target power information system, and updates the state vector based on the state data; determines a path correction factor based on the state data and the corresponding expected state information, and the execution feedback information and the corresponding expected execution information; when the path correction factor meets a preset replanning trigger condition, replans the executable attack path based on the updated state vector.

[0029] The aforementioned dynamic attack path planning method for power information systems obtains execution feedback information and target power information system status data after the attack action command is issued. Based on the status data, the state vector is updated. Based on the status data and the corresponding expected state information, as well as the execution feedback information and the corresponding expected execution information, a path correction factor is determined. When the path correction factor meets the replanning trigger condition, the executable attack path is replanned based on the updated state vector. This allows the attack path to be dynamically adjusted according to changes in the target power information system status and attack action execution during the test, thereby improving the attack path's adaptability to dynamic test environments. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating a dynamic planning method for attack paths in a power information system, as shown in one embodiment.

[0032] Figure 2 This is a flowchart illustrating a dynamic planning method for attack paths in a power information system, as shown in an example embodiment.

[0033] Figure 3 This is an internal structural diagram of a computer device in an example embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] In one embodiment, such as Figure 1 As shown, a dynamic planning method for attack paths in a power information system is provided, the method comprising the following steps:

[0036] Step 101: Obtain multi-dimensional state data of the target power information system, and construct the state vector of the target power information system based on the multi-dimensional state data;

[0037] Optionally, the multidimensional state data is cleaned, deduplicated, missing value imputed, and core features extracted, and then normalized to the [0,1] interval to generate a state vector. Among these, a 3D model is used. The principle is to identify outliers; missing values ​​are filled using the average of similar assets; features are selected based on the Pearson correlation coefficient, retaining only core features whose correlation with path success rate is greater than a preset threshold; the preset threshold can be 0.6. It is understood that the value of the preset threshold can be adjusted as needed.

[0038] Step 102: Determine the attack action sequence based on the state vector, and determine the executable attack path based on the attack action sequence;

[0039] Step 103: Determine the attack action command to be issued based on the executable attack path, and issue the attack action command to the test execution terminal;

[0040] Step 104: Receive the execution feedback information corresponding to the attack action command returned by the test execution terminal, as well as the status data of the target power information system, and update the status vector based on the status data;

[0041] Step 105: Based on the status data and the corresponding expected status information, as well as the execution feedback information and the corresponding expected execution information, determine the path correction factor;

[0042] Step 106: When the path correction factor meets the preset replanning trigger condition, the executable attack path is replanned based on the updated state vector.

[0043] In the aforementioned dynamic planning method for attack paths in power information systems, after the attack action command is issued, execution feedback information and the state data of the target power information system are obtained. The state vector is updated based on the state data. Based on the state data and the corresponding expected state information, as well as the execution feedback information and the corresponding expected execution information, a path correction factor is determined. When the path correction factor meets the replanning trigger condition, the executable attack path is replanned based on the updated state vector. This allows the attack path to be dynamically adjusted according to changes in the state of the target power information system and the execution of the attack action during the test, thereby improving the adaptability of the attack path to the dynamic test environment.

[0044] In one example embodiment, obtaining multi-dimensional status data of the target power information system includes: performing asset probing on the authorized target network segment, identifying online devices through ARP scanning and ICMP requests, and recording device IP, MAC, asset type, and security zone to form asset attribute data; performing TCP / UDP port probing on online devices based on the asset list to obtain port open status, and identifying service type and version by sending probe packets to open ports and matching service response feature databases to form network service data; matching service version and asset type with a preset vulnerability database to obtain vulnerability number, risk level, and exploitability status to form security risk data; obtaining alarm, blocking, and rate limiting information from firewalls, IDS, or IPS through the defense device log interface to form defense status data; obtaining CPU, memory, disk utilization, and concurrent connection count through SNMP or authorized remote methods, and obtaining network latency, packet loss rate, and test traffic bandwidth ratio through low-frequency network probing to form operating status and network performance data; and obtaining access permission data.

[0045] In one example implementation, asset detection is performed on the authorized target network segment. The target network segment 10.0.0.0 / 24 is scanned, identifying 32 power assets, categorized as 12 servers, 8 industrial control devices, 6 smart terminals, 4 communication gateways, and 2 database servers. Open ports such as 22, 80, 443, 6379, 5432, 102, and 502 (power-specific ports) are scanned. CentOS 7.6, Tomcat 9.0, Redis, PostgreSQL, and power dispatching software V5.0 are matched. The CNNVD vulnerability database is accessed, identifying 3 high-risk vulnerabilities, 5 medium-risk vulnerabilities, and 4 low-risk vulnerabilities. Bypassing the defense device log interface, it is obtained that the firewall has no alarms, IDS rules are enabled, and there is no blocking or flow restriction behavior. CPU utilization is 25%, memory utilization is 32%, and network latency is 18ms, packet loss rate is 0.005%, and bandwidth usage is 8%, obtained through low-frequency network detection.

[0046] In one embodiment, determining the attack action sequence based on the state vector includes: determining multiple candidate attack actions in the system state represented by the state vector based on a pre-built Markov decision process model; determining the action value corresponding to each candidate attack action using a pre-trained decision model; and determining the attack action sequence based on the action value corresponding to each candidate attack action.

[0047] This embodiment provides a basis for determining the sequence of attack actions.

[0048] In one embodiment, the Markov decision process model includes a state space, an action space, a state transition probability function, a multi-objective weighted reward function, and a discount factor. Each system state in the state space is represented by a corresponding state vector, and the action space includes multiple pre-set attack actions targeting the target power information system. The state transition probability function determines the probability of transitioning to the next system state after executing a candidate attack action in the corresponding system state. The multi-objective weighted reward function determines the reward value for executing a candidate attack action in the corresponding system state, and the discount factor adjusts the weight of the reward value corresponding to the subsequent system state.

[0049] Optionally, the Markov decision process model is a quintuple. Characterization, in which, For state space, For the action space, Let be the state transition probability function. For a multi-objective weighted reward function, This is the discount factor.

[0050] Optionally, motion space This includes eight types of attack actions within the scope of the power penetration testing authorization. ={a1,a2,a3,a4,a5,a6,a7,a8}, where a1 represents port scanning, a2 represents service probing, a3 represents vulnerability verification, a4 represents password guessing, a5 represents privilege escalation, a6 represents lateral movement, a7 represents vulnerability combination exploitation, and a8 represents vulnerability chain exploitation.

[0051] Optionally, based on historical penetration test data and interactive data from the target power information system simulation environment, the state transition probability function in the Markov decision process model is determined and updated. According to the formula Using the state transition probability function It can be determined in the first The system is in a state at any given time. And execute candidate attack actions Under the conditions, the first The time system transitions to the state. probability The value range is [0,1].

[0052] Optionally, the multi-objective weighted reward function is used to determine the reward value for performing a candidate attack action in the corresponding system state, and can be expressed as:

[0053] ;

[0054] in, , , , as well as The corresponding weights have a range of values. ,and ; Rewards for vulnerability discovery can be determined based on the number of newly discovered high-, medium-, and low-risk vulnerabilities. The reward for changes in permissions can be determined based on the difference between the current permission level and the previous permission level. The reward is for concealment and can be determined based on the number of defense alerts. Rewards for business operations can be determined based on system load, network latency, and packet loss rate. The path efficiency bonus can be determined based on the number of steps in the current attack path.

[0055] Optionally, , , , as well as The values ​​are 0.3, 0.2, 0.2, 0.2, and 0.1 in sequence. It's understandable that the weight values ​​can be adjusted according to actual needs. ,in, , as well as These are the number of newly discovered vulnerabilities categorized as high-, medium-, and low-risk, respectively. ,in, and These represent the current and previous permission levels, respectively. ,in, To prevent the number of alarms, Option 3 is acceptable. ,in, The current load of the target power information system. To preset the maximum load, For the current network latency, This represents the current packet loss rate. ,in, This represents the number of steps in the current attack path. It can be set to 20. This is understandable. and The value can be set according to specific needs.

[0056] Optionally, discount factor This is used to adjust the weight of the reward value corresponding to the subsequent system state, so as to take into account both the current reward and the reward corresponding to the subsequent state in the attack action decision-making process. A value of 0.95 is acceptable. The value can be adjusted according to actual needs.

[0057] In this embodiment, by setting a state space, action space, state transition probability function, multi-objective weighted reward function, and discount factor in the Markov decision process model, a comprehensive evaluation of candidate attack actions can be achieved, providing a decision basis for determining the attack action sequence.

[0058] In one example implementation, a Deep Q-Network (DQN) is used as the decision model to determine the action value corresponding to each candidate attack action in the current state vector. The DQN's input layer receives a 64-dimensional normalized state vector; the feature extraction layer consists of three fully connected layers with 256, 128, and 64 neurons respectively, employing the ReLU activation function; the output layer includes eight output nodes, each corresponding to the Q-value of one of the eight attack actions in the action space. The action value function is used to characterize the action value in the current state of the system. Execute candidate attack actions The corresponding long-term cumulative rewards.

[0059] During training, historical states, actions, rewards, and next state data are stored, and a target network with the same structure as the current network is set. A loss function is set to minimize the value error between the current network and the target network to achieve stable model training.

[0060] The current network parameters are updated using gradient descent, and periodically synchronized to the target network. The learning rate can be set to 1e-4, and the target network is synchronized every 100 steps. At the same time, an ε-greedy strategy is used for action selection, with ε gradually decreasing from 1.0 at the beginning of training to 0.01 with a decay coefficient of 0.995, in order to balance the exploration of candidate attack actions and the selection based on action value.

[0061] In one embodiment, the state vector includes asset attribute characteristics, network service characteristics, security risk characteristics, access permission characteristics, defense status characteristics, and operational status characteristics.

[0062] Optionally, asset attribute characteristics include, but are not limited to, power asset type, operating system and version; network service characteristics include, but are not limited to, open ports, service names and versions; security risk characteristics include, but are not limited to, vulnerability characteristics; access permission characteristics include, but are not limited to, permission levels; defense status characteristics include, but are not limited to, defense device status; and operating status characteristics include, but are not limited to, business load and network performance.

[0063] Optionally, the state vector is represented as =(Asset,OS,Port,Service,Vuln,Priv,Def,Load,Net), where Asset represents the type of power asset and can be encoded as 0, 1, 2, 3, and 4, corresponding to server, industrial control equipment, smart terminal, communication gateway, and database server, respectively; OS represents the operating system and version, which is normalized to a value in the range [0,1]; Port represents the list of open ports, mapped to a binary feature vector; Service represents the service name and version, quantized as a standardized value; Vuln represents the vulnerability characteristics, including the vulnerability risk level and exploitability status, with the vulnerability risk level encoded as 1, 2, and 4. 3 represents low risk, medium risk, and high risk, respectively. The exploitable status can be encoded as 0 and 1, corresponding to unexploitable and exploitable, respectively. Priv represents the permission level and can be encoded as 0, 1, 2, 3, and 4, corresponding to anonymous, visitor, ordinary user, administrator, and system permissions, respectively. Def represents the status of the defense device and can be encoded as 0, 1, 2, 3, and 4, corresponding to no alarm, minor alarm, severe alarm, blocking, and rate limiting, respectively. Load represents the service load, including CPU utilization, memory utilization, and concurrent connections, normalized to [0,1] values. Net represents network performance, including network latency, packet loss rate, and bandwidth ratio, normalized to [0,1] values.

[0064] This embodiment enables the current state of the target power information system to be represented from multiple different dimensions, providing a foundation for determining the attack action sequence based on the state vector and subsequent path replanning.

[0065] In one embodiment, determining an executable attack path based on an attack action sequence includes: generating an initial attack path based on the attack action sequence, and performing constraint verification on the initial attack path based on preset power service constraints to determine the executable attack path.

[0066] Optionally, attack steps are concatenated according to the sequence of attack actions, and the target asset and execution conditions of each step are labeled. The total number of steps and the estimated execution time are calculated to generate an initial attack path containing steps, targets, conditions, and execution time. During the generation of the initial attack path, the execution order of actions is arranged to avoid step conflicts.

[0067] Optionally, the steps and constraints in the initial attack path are compared item by item according to the power business constraints, and specific violations are marked as non-compliant.

[0068] This embodiment allows for the screening of whether an attack path meets power service constraints before the attack action command is issued, thus avoiding the direct use of initial attack paths that do not meet the corresponding constraints as executable attack paths.

[0069] In one embodiment, power service constraints include network performance constraints, service security constraints, permission compliance constraints, and behavior concealment constraints. Among them, network performance constraints are used to determine whether the initial attack path meets preset network performance requirements, service security constraints are used to determine whether the service access and testing behaviors involved in the initial attack path meet preset service security requirements, permission compliance constraints are used to determine whether the attack actions in the initial attack path meet preset permission compliance requirements, and behavior concealment constraints are used to determine whether the initial attack path meets preset behavior concealment requirements.

[0070] Optionally, network performance constraints include, but are not limited to, network latency constraints, packet loss rate constraints, and test traffic bandwidth usage constraints; business security constraints include, but are not limited to, power security area access constraints, scanning behavior constraints, and attack action execution interval constraints; permission compliance constraints include, but are not limited to, authorized action scope constraints, privilege escalation constraints, and unauthorized operation constraints; and behavior concealment constraints include, but are not limited to, defense alarm count constraints, connection frequency constraints, and malicious feature injection constraints.

[0071] This embodiment establishes a multi-dimensional constraint verification basis for power business scenarios.

[0072] In one example implementation, network performance constraints include: network latency ≤ 30ms, network packet loss rate ≤ 0.01%, and penetration traffic bandwidth ratio ≤ 10%; business security constraints include: prohibiting access to security zone I / II real-time control services, prohibiting high-volume, high-frequency scanning, and action execution interval ≥ 100ms; permission compliance constraints include: only executing authorized actions, following the principle of least privilege escalation, and prohibiting unauthorized operations; and behavior concealment constraints include: defense alarm count ≤ 3 times, connection frequency ≤ 5 times / second, and no malicious feature injection.

[0073] In one embodiment, determining the path correction factor based on state data and corresponding expected state information, and execution feedback information and corresponding expected execution information, includes: determining a state deviation based on the difference between the state data and the corresponding expected state information; determining an execution deviation based on the difference between the execution feedback information and the corresponding expected execution information; and determining the path correction factor based on the state deviation and the execution deviation.

[0074] Optionally, the execution deviation includes the action deviation; the action deviation is determined based on the difference between the actual action information in the execution feedback information and the expected action information in the expected execution information. A path correction factor is determined based on the state deviation and the action deviation.

[0075] Optionally, the formula for calculating the path correction factor is as follows:

[0076]

[0077] in, This is the path correction factor; Δ is the smoothing coefficient. Δ is the state deviation. = ,in, Represents the real-time state. Characterizes the expected state; This is the amount of motion deviation. ,in, Representing actual actions, It represents the expected action.

[0078] Optionally, It can be set to 0.7. This means the value can be set according to actual needs.

[0079] Optionally, when the path correction factor is greater than or equal to a preset replanning threshold, the path correction factor is determined to meet the preset replanning trigger condition; when the path correction factor is less than the preset replanning threshold, the original path is continued to be executed.

[0080] Optionally, the preset replanning threshold can be set to 0.3. This value can be adjusted according to actual needs.

[0081] Optionally, replanning the executable attack path based on the updated state vector includes: redetermining the attack action sequence based on the updated state vector, and updating the executable attack path according to the redetermined attack action sequence.

[0082] Optionally, the executable attack path can be partially updated based on the redefined attack action sequence, thereby improving efficiency without changing the overall path framework.

[0083] This embodiment provides a quantitative basis for determining whether an attack path replanning is triggered.

[0084] In one example implementation, an initial attack path is constructed based on the state vector of the target power information system. This path sequentially includes asset detection, port scanning, vulnerability verification, weak password guessing, privilege escalation, lateral movement, vulnerability chain exploitation, and target achievement. Specifically, the attack path is as follows: Tomcat vulnerability verification and weak password cracking are performed on port 443 of the 10.0.0.10 server; after escalating to administrator privileges, lateral movement is performed to the 10.0.0.20 database server, and a high-risk vulnerability chain exploitation is performed until the target is achieved. This path consists of 9 steps, with an execution time of 128 seconds, and no defense alarms or business disruptions are generated. During execution, if the original path triggers a minor IDS alarm during the privilege escalation, lateral movement, and vulnerability exploitation phases, and the state deviation Δ at this time... The path correction factor is 0.31. The value was 0.36, which is greater than the preset replanning threshold of 0.3. Therefore, replanning was performed based on the updated state vector. The corrected path included vulnerability verification, privilege downgrading, lateral movement, and vulnerability exploitation in sequence. No alarms or business disruptions were generated during the execution, and the path execution was successfully completed.

[0085] In one embodiment, after receiving execution feedback information and status data, the dynamic planning method for attack paths in power information systems further includes: updating the model parameters of the decision model based on the execution feedback information and status data, so that the updated decision model can be used to determine the subsequent attack action sequence.

[0086] Optionally, after issuing the attack action command to the test execution terminal, the power information system attack path dynamic planning method further includes: obtaining network bandwidth data during the execution of the attack action command by the test execution terminal, and performing permission verification on the attack action based on a preset authorization range; pausing the issuance of the attack action command when the network bandwidth data exceeds a preset bandwidth threshold; controlling the test execution terminal to terminate the execution of the attack action when it is determined from the permission verification result that the attack action exceeds the authorization range; and recording the action issuance time, execution object, action, attack traffic after rate limiting, and all operational behaviors.

[0087] Optionally, after determining or replanning the executable attack path based on the updated state vector, the test process data is organized and stored. Specifically, according to the execution order of the attack actions, the executable attack paths, the execution feedback information corresponding to each attack action, and the execution time are organized, and key vulnerabilities and permission change nodes in the path are marked, generating an attack path report containing path steps and execution results; based on the execution feedback information, the vulnerabilities that were actually successfully exploited are extracted, and the exploitation conditions, trigger results, and risk levels corresponding to each vulnerability are recorded, forming a vulnerability exploitation sequence report according to the preset risk levels; the logs generated throughout the process are classified and stored according to timestamps and processing stages to support subsequent querying and tracing, wherein the entire process includes state data collection, state vector construction, attack action determination, attack command issuance, execution feedback reception, and path replanning, etc.; based on the executable attack paths and corresponding execution results, the path execution status and vulnerability test results are statistically analyzed, and a test evaluation report of the target power information system is formed by combining the identified vulnerabilities and system state information.

[0088] Optionally, the test evaluation report may include, but is not limited to, a test overview, a vulnerability list, risk analysis, hardening recommendations, and compliance conclusions.

[0089] This embodiment enables closed-loop updates of the decision model driven by execution feedback information and status data, allowing subsequent attack decisions to continuously utilize the feedback and status information obtained during the preceding execution process.

[0090] In one example embodiment, multidimensional state data is acquired, including asset attribute data, network service data, security risk data, access permission data, defense status data, and operational status data of the target power information system. The acquired data is cleaned, deduplicated, imputed for missing values, and normalized to construct a state vector. A Markov decision process model, including a state space, action space, state transition probability function, multi-objective weighted reward function, and discount factor, is initialized. Constraint parameters representing preset power business constraints are loaded, and a pre-trained DQN decision model is loaded. Based on the current state vector, multiple candidate attack actions are determined. The DQN decision model determines the action value corresponding to each candidate attack action, and accordingly, the attack action sequence and initial attack path are determined. The initial attack path is then determined based on the loaded constraint parameters. Constraint verification is performed to determine executable attack paths. Attack action commands are issued to the test execution terminal based on these paths, and execution feedback information and state data of the target power information system are received, updating the state vector accordingly. The state deviation between the state data and the corresponding expected state information, as well as the execution deviation between the execution feedback information and the corresponding expected execution information, are calculated, and a path correction factor is determined based on these two factors. When the path correction factor meets the preset replanning trigger condition, the executable attack path is replanned based on the updated state vector; otherwise, the current path continues to be executed. The DQN decision model parameters are updated based on the execution feedback information and state data for determining the subsequent attack action sequence. Corresponding test results are generated based on the execution feedback information and the state data of the target power information system, and relevant data during the test process is recorded and stored.

[0091] In one example embodiment, a dynamic planning method for attack paths in a power information system is provided. Figure 2 The flowchart of this method includes the following steps:

[0092] The process involves acquiring multidimensional state data of the target power information system, cleaning, deduplicating, imputing missing values, extracting core features, and normalizing the data to generate a state vector for the target power information system. This state vector includes asset attribute features, network service features, security risk features, access permission features, defense status features, and operational status features. Based on a pre-built Markov decision process model, multiple candidate attack actions are identified within the system states represented by the state vector. A pre-trained decision model is used to determine the action value corresponding to each candidate attack action, and an attack action sequence is determined based on these action values. The Markov decision process model includes a state space, action space, state transition probability function, multi-objective weighted reward function, and discount factor. An initial attack path is generated based on the attack action sequence and a pre-defined power business agreement. The system performs constraint verification on the initial attack path to determine the executable attack path. Power business constraints include network performance constraints, business security constraints, permission compliance constraints, and behavioral concealment constraints. Based on the executable attack path, it determines the attack action command to be issued and sends it to the test execution terminal. It receives the execution feedback information corresponding to the attack action command returned by the test execution terminal, as well as the status data of the target power information system, and updates the state vector based on the status data. It determines the state deviation based on the difference between the status data and the corresponding expected state information. It determines the action deviation based on the difference between the actual action information in the execution feedback information and the expected action information in the expected execution information. It determines the path correction factor based on the state deviation and action deviation. When the path correction factor is greater than or equal to a preset replanning threshold, it replans the executable attack path based on the updated state vector.

[0093] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0094] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores multi-dimensional state data and other data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a dynamic programming method for attack paths in a power information system.

[0095] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0096] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0097] The system acquires multidimensional state data of the target power information system and constructs a state vector based on this data. It then determines an attack action sequence based on the state vector and an executable attack path according to that sequence. Based on the executable attack path, it determines the attack action command to be issued and sends it to the test execution terminal. The system receives execution feedback information corresponding to the attack action command returned by the test execution terminal, as well as the state data of the target power information system, and updates the state vector based on this data. Based on the state data and corresponding expected state information, and the execution feedback information and corresponding expected execution information, it determines a path correction factor. When the path correction factor meets a preset replanning trigger condition, it replans the executable attack path based on the updated state vector.

[0098] In one embodiment, determining the attack action sequence based on the state vector includes: determining multiple candidate attack actions in the system state represented by the state vector based on a pre-built Markov decision process model; determining the action value corresponding to each candidate attack action using a pre-trained decision model; and determining the attack action sequence based on the action value corresponding to each candidate attack action.

[0099] In one embodiment, the Markov decision process model includes a state space, an action space, a state transition probability function, a multi-objective weighted reward function, and a discount factor. Each system state in the state space is represented by a corresponding state vector, and the action space includes multiple pre-set attack actions targeting the target power information system. The state transition probability function determines the probability of transitioning to the next system state after executing a candidate attack action in the corresponding system state. The multi-objective weighted reward function determines the reward value for executing a candidate attack action in the corresponding system state, and the discount factor adjusts the weight of the reward value corresponding to the subsequent system state.

[0100] In one embodiment, the state vector includes asset attribute characteristics, network service characteristics, security risk characteristics, access permission characteristics, defense status characteristics, and operational status characteristics.

[0101] In one embodiment, determining an executable attack path based on an attack action sequence includes: generating an initial attack path based on the attack action sequence, and performing constraint verification on the initial attack path based on preset power service constraints to determine the executable attack path.

[0102] In one embodiment, power service constraints include network performance constraints, service security constraints, permission compliance constraints, and behavior concealment constraints. Among them, network performance constraints are used to determine whether the initial attack path meets preset network performance requirements, service security constraints are used to determine whether the service access and testing behaviors involved in the initial attack path meet preset service security requirements, permission compliance constraints are used to determine whether the attack actions in the initial attack path meet preset permission compliance requirements, and behavior concealment constraints are used to determine whether the initial attack path meets preset behavior concealment requirements.

[0103] In one embodiment, determining the path correction factor based on state data and corresponding expected state information, and execution feedback information and corresponding expected execution information, includes: determining a state deviation based on the difference between the state data and the corresponding expected state information; determining an execution deviation based on the difference between the execution feedback information and the corresponding expected execution information; and determining the path correction factor based on the state deviation and the execution deviation.

[0104] In one embodiment, after receiving execution feedback information and status data, the processor, when executing the computer program, further implements the step of updating the model parameters of the decision model based on the execution feedback information and status data, so that the updated decision model can be used to determine the subsequent attack action sequence.

[0105] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0106] The system acquires multidimensional state data of the target power information system and constructs a state vector based on this data. It then determines an attack action sequence based on the state vector and an executable attack path according to that sequence. Based on the executable attack path, it determines the attack action command to be issued and sends it to the test execution terminal. The system receives execution feedback information corresponding to the attack action command returned by the test execution terminal, as well as the state data of the target power information system, and updates the state vector based on this data. Based on the state data and corresponding expected state information, and the execution feedback information and corresponding expected execution information, it determines a path correction factor. When the path correction factor meets a preset replanning trigger condition, it replans the executable attack path based on the updated state vector.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0109] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0110] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A dynamic programming method for attack paths in a power information system, characterized in that, include: Acquire multidimensional state data of the target power information system, and construct the state vector of the target power information system based on the multidimensional state data; The attack action sequence is determined based on the state vector, and the executable attack path is determined based on the attack action sequence. The attack action command to be issued is determined based on the executable attack path, and the attack action command is issued to the test execution terminal. Receive the execution feedback information corresponding to the attack action instruction returned by the test execution terminal, as well as the status data of the target power information system, and update the status vector based on the status data; Based on the state data and the corresponding expected state information, as well as the execution feedback information and the corresponding expected execution information, a path correction factor is determined; When the path correction factor meets the preset replanning trigger condition, the executable attack path is replanned based on the updated state vector.

2. The method according to claim 1, characterized in that, The step of determining the attack action sequence based on the state vector includes: Based on a pre-constructed Markov decision process model, multiple candidate attack actions are determined for the system state represented by the state vector. The action value corresponding to each candidate attack action is determined using a pre-trained decision model, and the attack action sequence is determined based on the action value corresponding to each candidate attack action.

3. The method according to claim 2, characterized in that, The Markov decision process model includes a state space, an action space, a state transition probability function, a multi-objective weighted reward function, and a discount factor. Each system state in the state space is represented by a corresponding state vector. The action space includes multiple pre-set attack actions targeting the target power information system. The state transition probability function determines the probability of transitioning to the next system state after executing the candidate attack action in the corresponding system state. The multi-objective weighted reward function determines the reward value for executing the candidate attack action in the corresponding system state. The discount factor adjusts the weights of the reward values ​​corresponding to subsequent system states.

4. The method according to claim 1, characterized in that, The state vector includes asset attribute characteristics, network service characteristics, security risk characteristics, access permission characteristics, defense status characteristics, and operational status characteristics.

5. The method according to claim 1, characterized in that, The step of determining an executable attack path based on the attack action sequence includes: generating an initial attack path based on the attack action sequence, and performing constraint verification on the initial attack path based on preset power service constraints to determine the executable attack path.

6. The method according to claim 5, characterized in that, The power service constraints include network performance constraints, service security constraints, permission compliance constraints, and behavior concealment constraints. Specifically, the network performance constraints determine whether the initial attack path meets preset network performance requirements; the service security constraints determine whether the service access and testing behaviors involved in the initial attack path meet preset service security requirements; the permission compliance constraints determine whether the attack actions in the initial attack path meet preset permission compliance requirements; and the behavior concealment constraints determine whether the initial attack path meets preset behavior concealment requirements.

7. The method according to claim 1, characterized in that, The determination of the path correction factor based on state data and corresponding expected state information, as well as execution feedback information and corresponding expected execution information, includes: The state deviation amount is determined based on the difference between the state data and the corresponding expected state information; The execution deviation is determined based on the difference between the execution feedback information and the corresponding expected execution information; The path correction factor is determined based on the state deviation and the execution deviation.

8. The method according to claim 2, characterized in that, After receiving the execution feedback information and the status data, the method further includes: updating the model parameters of the decision model based on the execution feedback information and the status data, so that the updated decision model can be used to determine the subsequent attack action sequence.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images