Method and related device for generating cyber-physical attack vectors for power monitoring systems

Abstract

This application discloses a method and related device for generating cyber-physical attack vectors for a power monitoring system, relating to the field of information security technology for power monitoring systems. The method comprises: establishing a simulation model of the power monitoring system, controlling the operation of the simulation model according to the actual operation of the power monitoring system, performing power flow calculations on the simulation model to obtain initial measurement values, performing state estimation based on the initial measurement values ​​to obtain an initial state, using the initial state as input, utilizing the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state, applying the action to the simulation model to obtain the next state and reward corresponding to the initial state, iterating continuously until the Q value converges, terminating the iteration, and selecting the action with the maximum reward as the cyber-physical attack vector. This application can improve the efficiency and accuracy of generating cyber-physical attack vectors.

Description

Technical Field

[0001] The present application relates to the field of information security technology for power monitoring systems, and in particular to a method and related device for generating information-physical attack vectors for power monitoring systems based on the GCN-DuelingDQN algorithm. Background Art

[0002] The power monitoring system is a system used to monitor and control the power production and supply process. Based on computer and network technologies, it integrates intelligent devices and underlying communication and data networks to achieve real-time data acquisition, switch status detection, and remote control of the power system. However, with the deep integration of power systems and information and communication technologies, power monitoring systems are facing increasingly severe cyber-physical attack threats. In recent years, numerous cybersecurity incidents targeting power monitoring systems have occurred. In these cybersecurity incidents, attackers launched coordinated cyber-physical attacks, paralyzing the power monitoring system and causing significant losses. Therefore, from an attack perspective, it is particularly important to study methods for generating covert cyber-physical attack vectors. This will help improve the resilience of power monitoring systems and promote innovation in security protection technologies.

[0003] There are two common approaches to generating cyber-physical attack vectors. The first involves an attacker launching a physical attack and then a cyber attack to obscure the current state, misleading the control center into making an erroneous decision. The second involves an attacker launching a cyber attack first and then a physical attack to obscure the current state, misleading the control center into making an erroneous decision. The combination of cyber and physical attacks poses greater risks to power monitoring systems than either attack alone. Currently, traditional generation methods for the second approach include those based on heuristic search and optimization algorithms, those based on mixed integer programming, and those based on relaxation and approximation methods. While these traditional generation methods have demonstrated some effectiveness in generating cyber-physical attack vectors for traditional power monitoring systems, the increasing complexity and intelligence of power monitoring systems have posed significant challenges in terms of efficiency and accuracy, making it difficult to accurately capture and simulate the generation and evolution of cyber-physical attack vectors in emerging attack scenarios. Therefore, there is an urgent need to develop more efficient and accurate generation methods to address the security challenges of emerging power monitoring systems. Summary of the Invention

[0004] The purpose of this application is to provide a method and related device for generating cyber-physical attack vectors for an electric power monitoring system, which can improve the efficiency and accuracy of generating cyber-physical attack vectors.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for generating a cyber-physical attack vector for a power monitoring system, the method comprising:

[0007] Establishing a simulation model of the power monitoring system, controlling the operation of the simulation model according to the actual operation of the power monitoring system, and performing power flow calculations on the simulation model to obtain initial measurement values; the simulation model and the power monitoring system have identical circuit topologies; the measurement values ​​include the active power of each node in the power monitoring system and the active power at both ends of each transmission line, including load nodes, generator nodes, and busbar nodes;

[0008] Performing state estimation based on the initial measurement values ​​to obtain an initial state; the initial state includes the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line, and the state includes normal and abnormal;

[0009] The initial state is used as input, and the GCN-Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state; the action includes a load redistribution attack action and a physical manipulation attack action; the load redistribution attack action includes a plurality of selected load nodes to be attacked and the corresponding tampered injection power of each load node to be attacked; the physical manipulation attack action includes a plurality of selected overloaded lines to be attacked and the corresponding tampered reactance parameter of each overloaded line to be attacked; the overloaded line is a transmission line that is overloaded due to the load redistribution attack action;

[0010] Applying the action to the simulation model to obtain a next state and a reward corresponding to the initial state;

[0011] Determine whether the Q value converges and obtain the judgment result;

[0012] If the judgment result is yes, then the iteration ends and the action with the maximum reward is selected as the cyber-physical attack vector;

[0013] If the judgment result is no, determine whether the current number of iterations reaches the preset number of iterations; if so, update the GCN-Dueling DQN algorithm to obtain an updated GCN-Dueling DQN algorithm, set the current number of iterations to 0, and use the updated GCN-Dueling DQN algorithm as the GCN-Dueling DQN algorithm for the next iteration, use the next state as the initial state of the next iteration, and return to the step of "taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state". If not, add 1 to the current number of iterations, use the next state as the initial state of the next iteration, and return to the step of "taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state".

[0014] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for generating an information-physical attack vector for the power monitoring system.

[0015] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for generating an information-physical attack vector for the power monitoring system.

[0016] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for generating an information-physical attack vector for the power monitoring system.

[0017] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0018] The present application provides a method and related device for generating cyber-physical attack vectors for an electric power monitoring system. First, a simulation model of the electric power monitoring system is established. The operation of the simulation model is controlled according to the actual operation of the electric power monitoring system. The simulation model is subjected to flow calculation to obtain an initial measurement value. Based on the initial measurement value, a state estimation is performed to obtain an initial state. The initial state is used as input, and the GCN-Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state. The action is applied to the simulation model to obtain the next state and reward corresponding to the initial state. The iteration is continued until the Q value converges, at which point the iteration ends and the action with the largest reward is selected as the cyber-physical attack vector. The present application introduces the Dueling DQN algorithm. By separating the state value function and the action advantage function, it not only reduces the number of Q values ​​that need to be calculated in each iteration, but also reduces the number of parameters and computational complexity, thereby improving computational efficiency. At the same time, the cyber-physical attack vector can be generated through the training process. Compared with the method of first training and then applying to generate the cyber-physical attack vector, the computational efficiency can be further improved. The introduction of the GCN model can process graph-structured data and extract useful feature information through the connection relationships between nodes. When data is insufficient, it can supplement and enhance the feature representation in the learning process through graph structure information, generate better quality cyber-physical attack vectors, and solve the problem that the generated cyber-physical attack vectors may be relatively single and cannot cover all potential threats, affecting the diversity and flexibility of attack strategies, making it more difficult to cope with changing defense mechanisms. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is an application environment diagram of a method for generating cyber-physical attack vectors for a power monitoring system provided in Example 1 of the present application.

[0021] Figure 2 A flowchart of a method for generating a cyber-physical attack vector for a power monitoring system provided in Example 1 of the present application.

[0022] Figure 3 A schematic diagram of the principles of a method for generating cyber-physical attack vectors for a power monitoring system provided in Example 1 of the present application.

[0023] Figure 4 This is a flowchart of the GCN-Dueling DQN algorithm provided in Example 1 of the present invention.

[0024] Figure 5 A schematic diagram of the structure of a computer device provided in Example 2 of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] Example 1

[0027] The method for generating cyber-physical attack vectors for a power monitoring system provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal communicates with the server via a network. The data storage system can store data that the server needs to process. The data storage system can be set up separately, integrated on the server, or located in the cloud or on other servers. The terminal can send a pending generation request (for generating a cyber-physical attack vector for the power monitoring system) to the server. After receiving the pending generation request, the server establishes a simulation model of the power monitoring system, controls the operation of the simulation model according to the actual operation of the power monitoring system, and performs power flow calculations on the simulation model to obtain initial measurement values. Based on the initial measurement values, the server performs state estimation to obtain an initial state. Using the initial state as input, the GCN-Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state. The action is applied to the simulation model to obtain the next state and reward corresponding to the initial state. The Q value is determined to determine whether it has converged to obtain a judgment result. If the judgment result is yes, the iteration ends and the action with the highest reward is selected as the cyber-physical attack vector. The server can feedback the obtained cyber-physical attack vector for the generation request to the terminal.

[0028] In addition, in some embodiments, the information-physical attack vector generation method of the power monitoring system can also be implemented independently by a server or a terminal. For example, the terminal can directly process the generation request to be processed, or the server can obtain the generation request to be processed from the data storage system and process the generation request to be processed.

[0029] The terminals may be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or as a cloud server.

[0030] With the development of computing power and distributed computing, deep reinforcement learning-based generation methods have become a research hotspot for generating cyber-physical attack vectors. For example, the DQN (Deep Q-Learning) algorithm is used to generate cyber-physical attack vectors. The algorithm is divided into two stages:

[0031] The first stage is the training of the DQN algorithm. The specific process includes: initializing the DQN algorithm parameters; because the agent knows nothing about the environment, it is necessary to initialize the Q table in the network environment with an initial value of 0. The information in the Q table represents the knowledge or experience that the agent continuously acquires during the learning process; starting iteration and determining the initial state; starting training and selecting a new action with probability ε, otherwise selecting the action that maximizes the Q value; evaluating the action; observing the reward and the next state; storing the record (state s t , action a t , reward r t , the next state s t+1 ) to the experience replay buffer pool D; randomly collect N records from the experience replay buffer pool D (state s t , action a t , reward r t , the next state s t+1 ); if s i+1 is the final state, then r i Assign value to y i Otherwise, y i Press y i =r i +γm ax a Q(s i+1 ,a;θ) calculation, γ is the learning rate; according to Minimize the loss function through gradient descent to update the DQN algorithm parameters θ.

[0032] The second stage is the generation of cyber-physical attack vectors. The specific process includes: loading the trained DQN algorithm parameters θ; starting iteration and obtaining the initial state; physically disconnecting a transmission line and updating the topology of the power monitoring system; removing a transmission line from the network layer and causing the power monitoring system to re-dispatch power generation; updating the power flow and evaluating whether the key transmission line is overloaded; calculating the Q value from the Q network and t =arg max a Q(s t ,a;θ) select the action, and after the iteration, the information physical attack vector is obtained.

[0033] However, this approach has several drawbacks. First, it relies heavily on large amounts of high-quality training data; insufficient data can lead to poor training results. Second, it consumes significant computing resources in large-scale power monitoring systems and has poor generalization capabilities, requiring retraining whenever the power monitoring system changes. Finally, the generated cyber-physical attack vectors may be relatively limited and fail to cover all potential threats, limiting the diversity and flexibility of attack strategies and making it more difficult to respond to changing defense mechanisms.

[0034] In order to solve the above problems, Figure 2 and Figure 3 As shown, this embodiment provides a method for generating an information-physical attack vector for an electric power monitoring system. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The following steps are used as an example to illustrate the server in the example:

[0035] Step S1, establish a simulation model of the power monitoring system, control the operation of the simulation model according to the actual operation of the power monitoring system, and perform power flow calculation on the simulation model to obtain initial measurement values; the circuit topology structure of the simulation model and the power monitoring system are exactly the same; the measurement values ​​include the active power of each node in the power monitoring system and the active power at both ends of each transmission line, and the nodes include load nodes, generator nodes and bus nodes.

[0036] Step S2, performing state estimation based on the initial measurement value to obtain an initial state; the initial state includes the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line, and the state includes normal and abnormal.

[0037] Step S3, taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state; the action includes a load redistribution attack action and a physical manipulation attack action; the load redistribution attack action includes a selected number of load nodes to be attacked and the tampered injection power corresponding to each load node to be attacked; the physical manipulation attack action includes a selected number of overloaded lines to be attacked and the tampered reactance parameter corresponding to each overloaded line to be attacked; the overloaded line is a transmission line that is overloaded due to the load redistribution attack action.

[0038] Step S4: Apply the action to the simulation model to obtain the next state and reward corresponding to the initial state.

[0039] Step S5: determine whether the Q value has converged and obtain a determination result.

[0040] Step S6: If the judgment result is yes, the iteration ends and the action with the maximum reward is selected as the cyber-physical attack vector.

[0041] Step S7: If the judgment result is no, determine whether the current number of iterations reaches the preset number of iterations; if so, update the GCN-Dueling DQN algorithm to obtain an updated GCN-Dueling DQN algorithm, set the current number of iterations to 0, and use the updated GCN-Dueling DQN algorithm as the GCN-Dueling DQN algorithm for the next iteration, use the next state as the initial state of the next iteration, and return to the step of "taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state". If not, add 1 to the current number of iterations, use the next state as the initial state of the next iteration, and return to the step of "taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state".

[0042] Implementing the above-mentioned steps S1 to S7, this embodiment uses the GCN (Graph Convolutional Network)-Dueling DQN algorithm to generate the information-physical attack vector of the power monitoring system, introduces the Dueling DQN algorithm, and focuses more on learning the state value and action advantage by separating the state value function and the action advantage function. Since the state value function outputs a scalar and the action advantage function outputs a vector with the same dimension as the action space, this design not only reduces the number of Q values ​​that need to be calculated in each iteration, but also reduces the number of parameters and computational complexity, thereby reducing the consumption of computing resources. In particular, in an environment with complex state space and action space such as a large-scale power monitoring system, computing efficiency can be improved. At the same time, the information-physical attack vector can be generated through the training process, which can further improve computing efficiency compared to the method of training first and then applying to generate the information-physical attack vector. The introduction of the GCN model can process graph-structured data and extract useful feature information through the connection relationships between nodes. In the case of insufficient data, it can use graph structure information to supplement and enhance the feature representation in the learning process, generate better-quality cyber-physical attack vectors, and solve the problem that the generated cyber-physical attack vectors may be relatively simple and unable to cover all potential threats, affecting the diversity and flexibility of attack strategies, making it more difficult to cope with changing defense mechanisms. At the same time, it solves the problem of high reliance on large amounts of high-quality training data and insufficient data leading to poor training results. Subsequently, by setting up an experience replay pool and simulating the operating states and attack scenarios of different power monitoring systems, diverse experiences are generated and pre-stored in the experience replay pool. This increase in data diversity helps to better generalize to unknown or new power monitoring system environments and improve generalization capabilities.

[0043] This embodiment can adopt the third-party library pandapower in Python, customize the simulate_and_add_measurement function through the existing functions in the pandapower library, use the customized simulate_and_add_measurement function to load the predefined power monitoring system, establish a simulation model of the power monitoring system, the circuit topology structure of the simulation model and the power monitoring system is exactly the same, control the operation of the simulation model according to the actual operation of the power monitoring system, and perform power flow calculation on the simulation model, and simulate and collect initial measurement values, which include the active power of each node in the power monitoring system and the active power at both ends of each transmission line. The nodes include load nodes, generator nodes and bus nodes. The bus nodes include transformer nodes and connecting nodes, etc. The transmission line is used to connect two nodes. The active power at both ends of the transmission line includes the active power at the first end of the transmission line and the active power at the second end of the transmission line.

[0044] After loading and running the current power monitoring system, this embodiment generates a noisy measurement value based on the power flow calculation results of the simulation model and the specified noise level. By adding random noise, the measurement error that may exist in the actual power monitoring system is simulated. That is, noise of the specified noise level is added to the measurement value obtained by the power flow calculation to obtain a noisy measurement value, and the noisy measurement value is used as the initial measurement value.

[0045] After obtaining the initial measurement value, this embodiment first performs system state estimation and bad data detection to verify the validity of the measurement data simulated before the attack (i.e., the initial measurement value). Specifically, the system state estimation algorithm and the bad data detection algorithm are further called. The system state estimation algorithm and the bad data detection algorithm are both functions in pandapower. Specifically, the system state estimation algorithm can use a state estimation algorithm based on weighted least squares method, and the bad data detection algorithm can use a chi-square test method. System state estimation is performed by the system state estimation algorithm, and bad data detection is performed by the bad data detection algorithm. If the system state estimation is successful and no bad data is detected, it indicates that the measurement data simulated before the attack is valid. Subsequently, state estimation can be further performed based on the initial measurement value. If the system state estimation is unsuccessful or bad data is detected, it indicates that the measurement data simulated before the attack is invalid. The initial measurement value is regenerated by adjusting the input parameters of the custom simulate_and_add_measurement function.

[0046] At this time, in S2, state estimation is performed based on the initial measurement value to obtain the initial state, which specifically includes:

[0047] (1) Taking the initial measurement value as input, the system state estimation algorithm is used to determine the initial system state quantity, which includes the voltage and phase angle of each node in the power monitoring system.

[0048] (2) Taking the initial system state as input, the bad data detection algorithm is used to determine the initial bad data.

[0049] (3) If the difference between the initial system state quantity and the normal system state quantity is less than the preset difference, and the number of initial bad data is 0, the state is estimated based on the initial measurement value to obtain the initial state.

[0050] This embodiment can use the GCN model in the GCN-Dueling DQN algorithm to perform state estimation on the initial measurement value, form state perception of the power monitoring system environment, and obtain an initial state. The initial state includes the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line. The states include normal and abnormal. In the initial state, the state of each node and the state of each transmission line are normal, and the attention weight remains unchanged during subsequent iterations. The method for generating the attention weight includes: using the GCN model to learn the characteristics of the power monitoring system, using the task encoder to obtain the task representation, calculating the attention weight based on the characteristics and task representation, thereby determining the attention weight of each node and each transmission line, and optimizing the state representation of the power monitoring system.

[0051] like Figure 4 As shown, in S2, state estimation is performed based on the initial measurement value to obtain the initial state, which specifically includes:

[0052] (1) Control the simulation model according to the preset task and perform power flow calculation on the simulation model to obtain the measurement value corresponding to the task. The task includes covering up the line overload.

[0053] (2) Using the measurement value corresponding to the task as input, the first feature is obtained using the first GCN model.

[0054] (3) Using the measurement value corresponding to the task as input, the second feature is obtained using the second GCN model.

[0055] (4) Based on the measurement value corresponding to the task and the initial measurement value, the system state vector is determined. The system state vector includes the coding value of each node in the power monitoring system and the coding value of each transmission line. The coding value is 1 when the measurement value changes and 0 when the measurement value does not change.

[0056] For each node, if the value of the node in the measurement value corresponding to the task is the same as the value of the node in the initial measurement value, then the code value of the node is 0; otherwise, the code value of the node is 1. For each transmission line, if the value of the transmission line in the measurement value corresponding to the task is the same as the value of the transmission line in the initial measurement value, then the code value of the transmission line is 0; otherwise, the code value of the transmission line is 1.

[0057] (5) Taking the system state vector as input, the task encoder is used to obtain the task representation.

[0058] Among them, the task encoder is a multilayer perceptron (MLP), which is used to automatically learn and extract task-related feature representations to obtain task representations.

[0059] (6) Use the Softmax function to process the first feature and the task representation to obtain the processed feature.

[0060] (7) Pooling the second feature and the processed feature (i.e. Figure 4 The attention weight of each node and each transmission line in the power monitoring system is determined by using WeightingPooling in

[15] .

[0061] (8) The state of each node and the state of each transmission line in the power monitoring system are set to normal, and the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line form the initial state.

[0062] This example first defines multiple parameters in the GCN-Dueling DQN algorithm, as follows:

[0063] (1) State, including the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line. The state of the node and the state of the transmission line are of two types: normal and abnormal.

[0064] (2) Action, representing the selectable action type, including two action types: tampering with the load node injection power and the overload line reactance parameters, and tampering with the load node injection power but not tampering with the overload line reactance parameters. This embodiment specifically defines a load redistribution attack action and a physical manipulation attack action. The load redistribution attack action includes a number of selected load nodes to be attacked and the tampered injection power corresponding to each load node to be attacked. The physical manipulation attack action includes a number of selected overload lines to be attacked and the tampered reactance parameters corresponding to each overload line to be attacked. The overload line is a transmission line that is overloaded due to the load redistribution attack action. The number of overload lines to be attacked can be 0, which is equivalent to not tampering with the overload line reactance parameters.

[0065] (3) Reward, which represents the reward obtained after selecting a certain action in a certain state.

[0066] (4) Q function table, which reflects the discounted cumulative return that can be expected by choosing a certain action in a certain state and following the strategy until the terminal state, and this representation is achieved by decomposing the state value function and the action advantage function.

[0067] (5) Attack strategy set P. The format of attack strategy set P is <action, pre, post>, which means: after applying action action to the environment of the power monitoring system, in the current state pre, the next state that can be reached by the cyber-physical attack is post.

[0068] (6) Learning rate α, which is a preset parameter.

[0069] (7) Experience replay pool, used to store the experience generated during the training process to update the network parameters.

[0070] This embodiment first completes the preparations for generating a cyber-physical attack vector. These preparations are specifically divided into parameter initialization and parameter input. Parameter initialization involves initializing the Q function table in the GCN-Dueling DQN algorithm by setting all values ​​to 0. Parameter input involves inputting the attack strategy set P and the learning rate α. Then, based on this initial state, the cyber-physical attack vector is constructed through a training process, which proceeds to S3.

[0071] In S3, the initial state is used as input and the GCN-Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state. Specifically, the following steps are performed: the initial state is used as input and the third GCN model is used to determine the first node feature corresponding to the initial state; the first node feature corresponding to the initial state is used as input and the Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state.

[0072] The GCN-Dueling DQN algorithm combines the GCN model and the Dueling DQN algorithm. The GCN model is a neural network that performs convolution operations on graph data. The Dueling DQN algorithm is an improvement on the traditional DQN algorithm. It consists of three parts: an input layer, a hidden layer, and an output layer. The input layer contains multiple system state nodes, the number of which is determined by the number of state variables. The Dueling DQN algorithm divides the Q-value function into two parts: a state-value function V and an action-advantage function A. The GCN-Dueling DQN algorithm outputs an action, its state value, and its action-advantage. The sum of the state value and the centralized action-advantage is the Q-value corresponding to the action. The centralized action-advantage is the action-advantage of each action minus the average action-advantage of all possible actions in that state. It should be noted that when calculating the sum of the state value and the centralized action advantage, since the state value is a scalar and the centralized action advantage is a vector, the state value and each element of the centralized action advantage are summed separately to obtain the Q value, and the final Q value is a vector.

[0073] In the GCN-Dueling DQN algorithm, the GCN model and Dueling DQN algorithm are integrated. Specifically, the GCN model extracts node features from the initial state and uses these features as input to the Dueling DQN algorithm. The Dueling DQN algorithm then uses these node features to perform Q-value decomposition and strategy selection, outputting actions and their corresponding Q-values. The action selection is based on the initial state, the nodes and transmission lines that may subsequently experience anomalies corresponding to the initial state, the action flags Sc and Sp, the attack strategy set P, the Q-function table, and the learning rate α. The nodes and transmission lines that may subsequently experience anomalies are actually part of the attack strategy set. Based on the attack strategy set, we can identify load nodes that may be subsequently tampered with and transmission lines that may be overloaded due to the tampering. Sc and Sp represent whether to execute the load redistribution attack and the physical manipulation attack, respectively. A value of 1 indicates execution of the attack action, while a value of 0 indicates non-execution. Through this approach, the GCN model and Dueling DQN algorithm can work together to improve the ability to generate cyber-physical attack vectors for power monitoring systems. After performing an action, the next state and reward returned by the environment are received. The GCN-Dueling DQN algorithm is then updated based on the initial state, action, next state, and reward.

[0074] When selecting an action, the GCN-Dueling DQN algorithm relies on the Q-value function of the current state, which is updated during training. To balance exploration and exploitation, the GCN-Dueling DQN algorithm adopts an ε-greedy search strategy. Specifically, it randomly explores new actions with a probability of ε and selects the action with the largest Q-value from the known attack strategy set with a probability of 1-ε to obtain the action corresponding to the initial state.

[0075] In this embodiment, the GCN-Dueling DQN algorithm enhances the perception of state variables by fusing the GCN model and the Dueling DQN algorithm, so that the decision-making actions have a certain degree of consistency and better quality.

[0076] In this embodiment, after applying the selected action to the power monitoring system environment, the current state of the power monitoring system environment is perceived, i.e., the next state is obtained. Feedback is then provided for the action, and a corresponding reward is given. In S4, the action is applied to the simulation model to obtain the next state and reward corresponding to the initial state. When calculating the reward, after applying the selected action to the simulation model, a power flow calculation is performed. The relevant parameters in the power flow calculation results are then analyzed to determine the effect of the cyber-physical attack and provide a corresponding reward.

[0077] In order to improve the algorithm's memory function and convergence effect, the reward settings are specifically divided into the following situations:

[0078] (1) When the attacker successfully masks the overloaded line and remains undetected, that is, the attacker successfully tampered with the load node to inject power and coordinated tampering with the overloaded line reactance parameters, so that the line overload situation is completely concealed and the dispatcher cannot find any abnormalities through conventional monitoring means, the maximum positive reward is given based on the economic dispatch cost, and the maximum positive reward value is 100.

[0079] (2) When the attacker launches a load redistribution attack and successfully evades the system state estimation and bad data detection, the power flow calculation results are analyzed to see whether they cause line overload. If they do, an appropriate positive reward is given based on the economic dispatch cost. The appropriate positive reward value is 20.

[0080] (3) When the attacker launches a load redistribution attack, if he fails to successfully evade the system state estimation and bad data detection, or successfully evades the system state estimation and bad data detection but does not cause line overload, he will be given a maximum negative reward based on the economic dispatch cost, and the maximum negative reward value is -100.

[0081] The formula for calculating returns is:

[0082]

[0083] In formula (1), R is the return; R ed is the dispatching cost, i.e., the economic dispatching cost; a, b, and c are all conditions; a represents that both the load redistribution attack action and the physical manipulation attack action are verified to be successful, and after the load redistribution attack action successfully causes line overload, the physical manipulation attack action successfully masks the line overload; b represents that the load redistribution attack action is verified to be successful, and the load redistribution attack action successfully causes line overload; c represents that the load redistribution attack action is not verified to be successful, or the load redistribution attack action is verified to be successful, but fails to cause line overload.

[0084]

[0085] In formula (2), N G is the total number of generator nodes in the power monitoring system; α i is the first power generation cost coefficient of the i-th generator node; is the active power of the i-th generator node; β i is the second power generation cost coefficient of the i-th generator node; i is the third power generation cost coefficient of the i-th generator node.

[0086] The specific methods for determining conditions a, b, and c in the return calculation formula include:

[0087] (1) Apply the load redistribution attack action to the simulation model to obtain the first post-attack model, and perform power flow calculation on the first post-attack model to obtain the measurement value after the first attack.

[0088] In this embodiment, after the selected load redistribution attack action is applied to the environment of the power monitoring system, power flow calculation is performed to obtain the measurement value after the first attack.

[0089] (2) Verify the load redistribution attack action based on the measurement value after the first attack to obtain a first verification result.

[0090] In this embodiment, based on the tampered measurement data (i.e., the measurement value after the first attack), a system state estimation algorithm is used to obtain the system state quantity after the network attack. The bad data detection algorithm is used to detect bad data in the tampered measurement data. If the tampered measurement data successfully evades the system state estimation and bad data detection, the power flow calculation is then performed.

[0091] At this time, the load redistribution attack action is verified based on the measurement value after the first attack to obtain a first verification result, which specifically includes: taking the measurement value after the first attack as input, and using the system state estimation algorithm to determine the system state quantity after the first attack; taking the system state quantity after the first attack as input, and using the bad data detection algorithm to determine the bad data after the first attack; if the difference between the system state quantity after the first attack and the normal system state quantity is less than the preset difference, and the amount of bad data after the first attack is 0, then the first verification result is verification passed; otherwise, the first verification result is verification failed.

[0092] (3) If the first verification result is verification failure, it is determined that the action belongs to condition c, and the reward corresponding to the action is further calculated using formula (1).

[0093] (4) If the first verification result is verification passed, the power flow calculation is performed on the first post-attack model to obtain the actual current value of each transmission line in the power monitoring system, and the transmission line whose actual current value is greater than the maximum rated current is recorded as an overloaded line.

[0094] This embodiment further analyzes the actual current value of each transmission line in the power flow calculation result. If the actual current value is greater than the maximum rated current of the line, the transmission line is determined to be an overloaded line, and all overloaded lines are grouped into an overloaded line set L.

[0095] (5) Determine whether the number of overloaded lines is 0; if so, determine that the action belongs to condition c, and further use formula (1) to calculate the reward corresponding to the action; if not, determine that the action belongs to condition b, and further use formula (1) to calculate the reward corresponding to the action.

[0096] (6) Applying the physical manipulation attack action to the first post-attack model to obtain a second post-attack model, and performing a power flow calculation on the second post-attack model to obtain the second post-attack measurement value.

[0097] In this embodiment, based on the overloaded line set L, the selected physical manipulation attack action is applied to the environment of the power monitoring system to mask the line overload caused by the load redistribution attack action, and power flow calculation is performed to obtain the measurement value after the second attack.

[0098] (7) Verify the physical manipulation attack action based on the measurement value after the second attack to obtain a second verification result.

[0099] In this embodiment, based on the tampered measurement data (i.e., the measurement value after the second attack), a system state estimation algorithm is used to estimate the system state after the physical attack, and a bad data detection algorithm is used to detect bad data in the tampered measurement data. If the tampered measurement data successfully evades the system state estimation and bad data detection, the power flow calculation is then performed.

[0100] At this time, the physical manipulation attack action is verified based on the measurement value after the second attack to obtain a second verification result, which specifically includes: taking the measurement value after the second attack as input, and using the system state estimation algorithm to determine the system state quantity after the second attack; taking the system state quantity after the second attack as input, and using the bad data detection algorithm to determine the bad data after the second attack; if the difference between the system state quantity after the second attack and the normal system state quantity is less than the preset difference, and the amount of bad data after the second attack is 0, then the second verification result is verification passed; otherwise, the second verification result is verification failed.

[0101] (8) If the second verification result is verification failure, it is determined that the action belongs to condition b, and the reward corresponding to the action is further calculated using formula (1); if the second verification result is verification success, it is determined that the action belongs to condition a, and the reward corresponding to the action is further calculated using formula (1).

[0102] In this embodiment, an experience replay pool is set for the GCN-Dueling DQN algorithm. It is the training sample library of the GCN-Dueling DQN algorithm, obtained by the agent continuously interacting with the environment for trial-and-error learning, solving the problem of overfitting in traditional reinforcement learning, and improving the convergence effect of the algorithm. The training process is mainly reflected in the process where the agent optimizes its strategy through continuous iteration and trial-and-error, including the entire process from state perception to action selection, action execution and feedback, policy update, and iteration and convergence judgment. The agent continuously interacts with the environment and updates its policy according to the received rewards until the Q value converges.

[0103] Among them, the setting of the experience in the experience replay pool is specifically as follows: <state, action, reward, state'>, where state is the current state, action is the action taken in the current state, reward is the corresponding reward after taking the action in the current state, and state' is the next state entered after executing the action. During the learning process, new experiences are constantly added to the experience replay pool. To ensure that the latest experience can be used to train the network, the update rule of the experience replay pool is set as the first-in, first-out principle.

[0104] In this embodiment, it is judged whether the Q value converges to obtain a judgment result. The Dueling DQN algorithm uses a loss function to measure the gap between the currently estimated Q value and the target Q value. When the value of the loss function tends to be stable in consecutive multiple iterations, that is, no significant change occurs, it indicates that the Q value converges. If the judgment result is yes, that is, the Q value converges, the iteration ends, and the action with the largest reward is selected as the cyber-physical attack vector. If the judgment result is no, that is, the Q value does not converge, it is judged whether the current iteration number reaches the preset iteration number; if so, the GCN-Dueling DQN algorithm is updated to obtain the updated GCN-Dueling DQN algorithm, the current iteration number is set to 0, and the updated GCN-Dueling DQN algorithm is used as the GCN-Dueling DQN algorithm for the next iteration, and the next state is used as the initial state for the next iteration, and the step of "using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state with the initial state as the input" is returned; if not, the current iteration number is incremented by 1, and the next state is used as the initial state for the next iteration, and the step of "using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state with the initial state as the input" is returned.

[0105] After obtaining the next state and reward corresponding to the initial state, the information-physical attack vector generation method of the power monitoring system of this embodiment also includes: combining the initial state and the action, next state and reward corresponding to the initial state into an experience, and storing the experience in an experience replay pool. The experience replay pool pre-stores the experience generated by simulating information-physical attacks on different power monitoring systems. At this time, the GCN-Dueling DQN algorithm is updated to obtain an updated GCN-Dueling DQN algorithm, specifically including: selecting part of the experience from the experience replay pool according to the first-in-first-out principle, and using part of the experience to update the GCN-Dueling DQN algorithm to obtain an updated GCN-Dueling DQN algorithm.

[0106] According to the reward, the Q value in the GCN-Dueling DQN algorithm is updated according to the following formula (3), and the network parameters of the Dueling DQN algorithm are further updated later:

[0107]

[0108] In formula (3), Q(state, action) is the Q value, state is the initial state, action is the action corresponding to the initial state; R(state, action) is the reward; α is the learning rate; state' is the next state; action' is the action corresponding to the next state.

[0109] This embodiment further uses two standard power monitoring system test cases, case 39 and case 118, for simulation and analysis. The results show that the method of this embodiment has excellent performance and can generate cyber-physical attack vectors efficiently and accurately.

[0110] This embodiment provides a method for generating cyber-physical attack vectors for a power monitoring system based on the GCN-Dueling DQN algorithm. This method first initiates a load redistribution cyber attack, then launches a physical manipulation attack to mask the actual overloaded lines caused by the cyber attack. This causes the control center to issue suboptimal dispatch instructions based on erroneous state estimation results, thereby affecting the normal operation of the power monitoring system. First, a custom simulate_and_add_measurement function is used to load and run a predefined power monitoring system, simulating measurement values ​​from collection nodes and transmission lines. State estimation is then performed on these measurements to form environmental awareness of the power monitoring system and obtain an initial state. Finally, based on this initial state, a cyber-physical attack vector is constructed through the training process of the GCN-Dueling DQN algorithm. This embodiment uses the GCN-DuelingDQN algorithm to improve the efficiency and scale of cyber-physical attack vector generation. It can generate cyber-physical attack vectors dynamically and in real time. The improved scale is due to the ability to select better actions during the iteration process, thereby speeding up iterations and reducing the number of iterations. By introducing the Dueling DQN algorithm, each learning process is no longer necessarily dependent on other learning scenarios. The GCN model is introduced to learn the node characteristics of the power monitoring system's operating status, achieving better training results and obtaining more optimized cyber-physical attack vectors.

[0111] The method of this embodiment has the following advantages:

[0112] (1) Customized generation of cyber-physical attack vectors based on the GCN-Dueling DQN algorithm can not only improve the efficiency and scale of cyber-physical attack vector generation, but also dynamically generate cyber-physical attack vectors in real time by setting the experience replay pool, thereby improving generalization ability.

[0113] (2) By introducing the GCN-Dueling DQN algorithm, a variety of action types can be selected according to different system states and attack targets, making the attack strategy more flexible and changeable, thereby quickly generating more covert attack strategies and increasing the difficulty of defense.

[0114] (3) After generating the cyber-physical attack vector of the power monitoring system, the attack and defense of the entire cyber-physical attack vector can be simulated through the GCN-Dueling DQN algorithm, and the cyber-physical attack vector can be constructed in the simulation.

[0115] The present application also provides an application scenario that applies the aforementioned method for generating cyber-physical attack vectors for a power monitoring system. Specifically, the method for generating cyber-physical attack vectors for a power monitoring system provided in this embodiment can be applied in a power monitoring system defense scenario. The power monitoring system defense scenario includes a processing link and a defense link. The processing link is used to generate cyber-physical attack vectors, and the defense link is used to develop corresponding defense measures based on the cyber-physical attack vectors to defend the power monitoring system. The method for generating cyber-physical attack vectors for a power monitoring system provided in this embodiment belongs to the processing link.

[0116] Example 2

[0117] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating an information-physical attack vector for a power monitoring system is implemented.

[0118] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0119] In an exemplary embodiment, a computer device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for generating an information-physical attack vector for the power monitoring system described in Example 1.

[0120] Example 3

[0121] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for generating a cyber-physical attack vector for the power monitoring system described in Example 1 is implemented.

[0122] Example 4

[0123] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for generating a cyber-physical attack vector for a power monitoring system described in Example 1.

[0124] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for generating cyber-physical attack vectors for a power monitoring system, characterized in that: The method for generating a cyber-physical attack vector for a power monitoring system includes: Establishing a simulation model of the power monitoring system, controlling the operation of the simulation model according to the actual operation of the power monitoring system, and performing power flow calculations on the simulation model to obtain initial measurement values; the simulation model and the power monitoring system have identical circuit topologies; the measurement values ​​include the active power of each node in the power monitoring system and the active power at both ends of each transmission line, including load nodes, generator nodes, and busbar nodes; Performing state estimation based on the initial measurement values ​​to obtain an initial state; the initial state includes the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line, and the state includes normal and abnormal; The initial state is used as input, and the GCN-Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state; the action includes a load redistribution attack action and a physical manipulation attack action; the load redistribution attack action includes a plurality of selected load nodes to be attacked and the corresponding tampered injection power of each load node to be attacked; the physical manipulation attack action includes a plurality of selected overloaded lines to be attacked and the corresponding tampered reactance parameter of each overloaded line to be attacked; the overloaded line is a transmission line that is overloaded due to the load redistribution attack action; Applying the action to the simulation model to obtain a next state and a reward corresponding to the initial state; Determine whether the Q value converges and obtain the judgment result; If the judgment result is yes, then the iteration ends and the action with the maximum reward is selected as the cyber-physical attack vector; If the judgment result is no, determine whether the current number of iterations reaches the preset number of iterations; if so, update the GCN-Dueling DQN algorithm to obtain an updated GCN-Dueling DQN algorithm, set the current number of iterations to 0, and use the updated GCN-Dueling DQN algorithm as the GCN-Dueling DQN algorithm for the next iteration, use the next state as the initial state of the next iteration, and return to the step of "taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state". If not, add 1 to the current number of iterations, use the next state as the initial state of the next iteration, and return to the step of "taking the initial state as input, and using the GCN-Dueling DQN algorithm to determine the action and Q value corresponding to the initial state".

2. The method for generating cyber-physical attack vectors for a power monitoring system according to claim 1, characterized in that: Based on the initial measurement value, the state is estimated to obtain the initial state, which specifically includes: Controlling the simulation model to run according to a preset task, and performing power flow calculation on the simulation model to obtain measurement values ​​corresponding to the task; the task includes masking a line overload; Using the measurement value corresponding to the task as input, obtaining a first feature using a first GCN model; Using the measurement value corresponding to the task as input, a second feature is obtained using a second GCN model; Determine a system state vector based on the measured value corresponding to the task and the initial measured value; the system state vector includes a code value of each node and a code value of each transmission line in the power monitoring system, where the code value is 1 when the measured value changes and is 0 when the measured value does not change; Taking the system state vector as input, obtaining a task representation using a task encoder; Processing the first feature and the task representation using a Softmax function to obtain a processed feature; Pooling the second feature and the processed feature to determine an attention weight of each node and an attention weight of each transmission line in the power monitoring system; The state of each node and the state of each transmission line in the power monitoring system are set to normal, and the state and attention weight of each node in the power monitoring system and the state and attention weight of each transmission line form the initial state.

3. The method for generating cyber-physical attack vectors for a power monitoring system according to claim 1, characterized in that: The formula for calculating the return is: Among them, R is the return; R ed is the dispatching cost; a, b, and c are all conditions; a represents that both the load redistribution attack action and the physical manipulation attack action are verified to be successful, and after the load redistribution attack action successfully causes line overload, the physical manipulation attack action successfully masks the line overload; b represents that the load redistribution attack action is verified to be successful, and the load redistribution attack action successfully causes line overload; c represents that the load redistribution attack action fails to be verified or passes the load redistribution attack action but fails to cause line overload. Among them, N G is the total number of generator nodes in the power monitoring system; α i is the first power generation cost coefficient of the i-th generator node; is the active power of the i-th generator node; β i is the second power generation cost coefficient of the i-th generator node; i is the third power generation cost coefficient of the i-th generator node.

4. The method for generating cyber-physical attack vectors for a power monitoring system according to claim 3, characterized in that: The specific methods for determining conditions a, b, and c in the return calculation formula include: Applying the load redistribution attack action to the simulation model to obtain a first post-attack model, and performing power flow calculation on the first post-attack model to obtain a first post-attack measurement value; Verifying the load redistribution attack action based on the measurement value after the first attack to obtain a first verification result; If the first verification result is verification failure, determining that the action belongs to condition c; If the first verification result is verification passed, then performing power flow calculation on the first post-attack model to obtain the actual current value of each transmission line in the power monitoring system, and recording the transmission line whose actual current value is greater than the maximum rated current as an overloaded line; Determine whether the number of overloaded lines is 0; if so, determine that the action belongs to condition c; if not, determine that the action belongs to condition b; Applying the physical manipulation attack action to the first post-attack model to obtain a second post-attack model, and performing power flow calculation on the second post-attack model to obtain a second post-attack measurement value; verifying the physical manipulation attack action based on the measurement value after the second attack to obtain a second verification result; If the second verification result is verification failure, it is determined that the action belongs to condition b; if the second verification result is verification success, it is determined that the action belongs to condition a.

5. The method for generating cyber-physical attack vectors for a power monitoring system according to claim 4, characterized in that: Based on the initial measurement value, the state is estimated to obtain the initial state, which specifically includes: Using the initial measurement value as input, the system state estimation algorithm is used to determine the initial system state quantity; the system state quantity includes the voltage and phase angle of each node in the power monitoring system; Taking the initial system state as input, the bad data detection algorithm is used to determine the initial bad data; If the difference between the initial system state quantity and the normal system state quantity is less than the preset difference, and the number of initial bad data is 0, then the state is estimated based on the initial measurement value to obtain the initial state; The load redistribution attack action is verified based on the measurement value after the first attack to obtain a first verification result, specifically including: Taking the measured value after the first attack as input, a system state estimation algorithm is used to determine the system state quantity after the first attack; Taking the system state after the first attack as input, a bad data detection algorithm is used to determine the bad data after the first attack; If the difference between the system state quantity after the first attack and the normal system state quantity is less than the preset difference, and the amount of bad data after the first attack is 0, then the first verification result is verification passed; otherwise, the first verification result is verification failed; The physical manipulation attack action is verified based on the measurement value after the second attack to obtain a second verification result, specifically including: Taking the measured value after the second attack as input, a system state estimation algorithm is used to determine the system state quantity after the second attack; Taking the system state after the second attack as input, a bad data detection algorithm is used to determine the bad data after the second attack; If the difference between the system state quantity after the second attack and the normal system state quantity is less than the preset difference, and the amount of bad data after the second attack is 0, the second verification result is verification passed; otherwise, the second verification result is verification failed.

6. The method for generating cyber-physical attack vectors for a power monitoring system according to claim 1, characterized in that: Using the initial state as input, the GCN-Dueling DQN algorithm is used to determine the action and Q value corresponding to the initial state, specifically including: Taking the initial state as input, determining a first node feature corresponding to the initial state using a third GCN model; The first node feature corresponding to the initial state is used as input, and the action and Q value corresponding to the initial state are determined using the Dueling DQN algorithm.

7. The method for generating cyber-physical attack vectors for a power monitoring system according to claim 1, characterized in that: After obtaining the next state and reward corresponding to the initial state, the information-physical attack vector generation method of the power monitoring system also includes: combining the initial state and the action, next state and reward corresponding to the initial state into an experience, and storing the experience in an experience replay pool; the experience replay pool pre-stores experience generated by simulating information-physical attacks on different power monitoring systems; at this time, the GCN-Dueling DQN algorithm is updated to obtain an updated GCN-Dueling DQN algorithm, specifically including: selecting part of the experience from the experience replay pool according to the first-in-first-out principle, and using part of the experience to update the GCN-Dueling DQN algorithm to obtain an updated GCN-Dueling DQN algorithm.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for generating an information-physical attack vector for an electric power monitoring system according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating an information-physical attack vector for an electric power monitoring system according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for generating an information-physical attack vector for an electric power monitoring system according to any one of claims 1 to 7 is implemented.

Images