Network security penetration testing method, device and computer equipment of power system

Abstract

The application relates to a network security penetration testing method and device of a power system and computer equipment, and relates to the technical field of network security testing. The method comprises the following steps: determining the network security penetration testing strategy of each test intelligent agent based on the network security penetration testing instruction of the power system; performing the network security penetration testing on the power system through the execution of the network security penetration testing strategy of each test intelligent agent; obtaining the observation information of each test intelligent agent for the power system in the process of the network security penetration testing; the observation information is obtained through the observation of the running state of the power system by the test intelligent agent; and the network security penetration testing strategy corresponding to each test intelligent agent is dynamically updated based on the preset reward function and the observation information of each test intelligent agent, so that the updated network security penetration testing strategy corresponding to each test intelligent agent is obtained. The method can effectively evaluate the network security performance of the power system.

Description

Technical Field

[0001] This application relates to the field of network security testing technology, and in particular to a network security penetration testing method, apparatus, computer equipment, computer-readable storage medium, and computer program product for a power system. Background Technology

[0002] Network security penetration testing is a method of evaluating the network security performance of a computer network by simulating hacker attacks.

[0003] In related technologies, testing strategies are typically developed based on human experience, and cybersecurity penetration tests are then conducted on the targets based on these strategies. However, for power systems, the topology of the power network and the operating status of the included power equipment change in real time. This makes it difficult for cybersecurity penetration tests based on human experience to effectively assess the cybersecurity performance of power systems. Summary of the Invention

[0004] Therefore, it is necessary to address the aforementioned technical problem of difficulty in effectively assessing the network security performance of power systems by providing a network security penetration testing method, apparatus, computer equipment, computer-readable storage medium, and computer program product for power systems that can effectively assess the network security performance of power systems.

[0005] Firstly, this application provides a cybersecurity penetration testing method for power systems, including:

[0006] Based on the cybersecurity penetration test instructions for the power system, determine the cybersecurity penetration test strategies for each test agent.

[0007] By executing the corresponding network security penetration testing strategy by each of the aforementioned test agents, network security penetration testing is performed on the power system.

[0008] During the network security penetration test, observation information of each test agent regarding the power system is acquired; the observation information is obtained through the test agents' observation of the power system's operating status.

[0009] Based on the preset reward function and the observation information of each test agent, the network security penetration testing strategy corresponding to each test agent is dynamically updated to obtain the updated network security penetration testing strategy corresponding to each test agent.

[0010] The updated network security penetration testing strategy corresponding to each of the test agents is used as the new network security penetration testing strategy for each of the test agents. The steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents are returned until the preset network security penetration testing termination conditions are met.

[0011] In one embodiment, the test agent includes at least a boundary detection agent;

[0012] The method further includes:

[0013] The boundary detection agent performs a non-intrusive scan of each boundary device of the power system's power network to determine the device risk value of each boundary device.

[0014] The boundary detection agent identifies target boundary devices whose risk values ​​are greater than or equal to the risk threshold among the boundary devices, and determines the boundary protection strength assessment result of the power network based on the risk values ​​of each boundary device.

[0015] In one embodiment, the test agent includes at least a protocol-specific agent;

[0016] The method further includes:

[0017] The protocol-specific intelligent agent captures and parses communication messages within the power network of the power system to determine the communication protocol corresponding to the power network.

[0018] The design flaws and vulnerabilities of the communication protocol are identified through the protocol-specific intelligent agent.

[0019] In one embodiment, the test agent includes at least an intranet penetration agent;

[0020] The method further includes:

[0021] The intranet penetration agent performs lateral movement attacks on target intranet devices in the power system's power network; the target intranet devices are those whose importance to the power system is greater than or equal to the importance value among the various intranet devices in the power network.

[0022] Sensitive data about the power system is obtained from the target intranet device through the lateral movement attack of the intranet penetration agent.

[0023] In one embodiment, the test agent includes at least a social engineering agent;

[0024] The method further includes:

[0025] The social engineering agent was used to conduct a simulated phishing attack on the power system.

[0026] The social engineering agent obtains the target operation logs within the power system; the target operation logs are the operation logs among the various operation logs within the power system used to record human operations in response to the simulated phishing attack.

[0027] In one embodiment, the test agent includes at least an emergency assessment agent;

[0028] The method further includes:

[0029] The emergency assessment agent monitors the operating status of the power system in real time, captures alarm information in the power system, and determines whether there is an emergency response action in response to the alarm information in the power system.

[0030] If the emergency assessment agent determines, based on the operating status of the power system, that the system risk value of the power system is greater than or equal to the system risk threshold, the cybersecurity penetration test on the power system will be terminated.

[0031] In one embodiment, the reward function includes a first reward function, a second reward function, and a third reward function; the first reward function is used to quantify the value of each test agent's attack on the power system for the security penetration test; the second reward function is used to constrain the impact of the network security penetration test on the stable operation of the power system; and the third reward function is used to optimize the collaboration efficiency of each test agent.

[0032] The dynamic updating of the network security penetration testing strategy executed by each test agent based on a preset reward function and the observation information of each test agent includes:

[0033] Based on the observation information of each of the test agents, the first function value of the network security penetration test under the first reward function, the second function value under the second reward function, and the third function value under the third reward function are determined.

[0034] The first function value, the second function value, and the third function value are weighted and fused to obtain the current total reward value of the network security penetration test.

[0035] Based on the total reward value, the network security penetration testing strategy corresponding to each of the test agents is dynamically updated.

[0036] Secondly, this application also provides a security penetration testing device for a power system, comprising:

[0037] The strategy determination module is used to determine the network security penetration testing strategy for each testing agent based on the network security penetration testing instructions for the power system.

[0038] The penetration testing module is used to perform network security penetration testing on the power system by executing the corresponding network security penetration testing strategies through each of the test agents.

[0039] An information observation module is used to acquire observation information of the power system by each test agent during the network security penetration test; the observation information is obtained by the test agents observing the operating status of the power system.

[0040] The strategy update module is used to dynamically update the network security penetration testing strategy corresponding to each of the test agents based on a preset reward function and the observation information of each of the test agents, so as to obtain the updated network security penetration testing strategy corresponding to each of the test agents.

[0041] The penetration testing module is further configured to use the updated network security penetration testing strategy corresponding to each of the test agents as the new network security penetration testing strategy for each of the test agents, and return the steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents, until the preset network security penetration testing termination conditions are met.

[0042] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0043] Based on the cybersecurity penetration test instructions for the power system, determine the cybersecurity penetration test strategies for each test agent.

[0044] By executing the corresponding network security penetration testing strategy by each of the aforementioned test agents, network security penetration testing is performed on the power system.

[0045] During the network security penetration test, observation information of each test agent regarding the power system is acquired; the observation information is obtained through the test agents' observation of the power system's operating status.

[0046] Based on the preset reward function and the observation information of each test agent, the network security penetration testing strategy corresponding to each test agent is dynamically updated to obtain the updated network security penetration testing strategy corresponding to each test agent.

[0047] The updated network security penetration testing strategy corresponding to each of the test agents is used as the new network security penetration testing strategy for each of the test agents. The steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents are returned until the preset network security penetration testing termination conditions are met.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0049] Based on the cybersecurity penetration test instructions for the power system, determine the cybersecurity penetration test strategies for each test agent.

[0050] By executing the corresponding network security penetration testing strategy by each of the aforementioned test agents, network security penetration testing is performed on the power system.

[0051] During the network security penetration test, observation information of each test agent regarding the power system is acquired; the observation information is obtained through the test agents' observation of the power system's operating status.

[0052] Based on the preset reward function and the observation information of each test agent, the network security penetration testing strategy corresponding to each test agent is dynamically updated to obtain the updated network security penetration testing strategy corresponding to each test agent.

[0053] The updated network security penetration testing strategy corresponding to each of the test agents is used as the new network security penetration testing strategy for each of the test agents. The steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents are returned until the preset network security penetration testing termination conditions are met.

[0054] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0055] Based on the cybersecurity penetration test instructions for the power system, determine the cybersecurity penetration test strategies for each test agent.

[0056] By executing the corresponding network security penetration testing strategy by each of the aforementioned test agents, network security penetration testing is performed on the power system.

[0057] During the network security penetration test, observation information of each test agent regarding the power system is acquired; the observation information is obtained through the test agents' observation of the power system's operating status.

[0058] Based on the preset reward function and the observation information of each test agent, the network security penetration testing strategy corresponding to each test agent is dynamically updated to obtain the updated network security penetration testing strategy corresponding to each test agent.

[0059] The updated network security penetration testing strategy corresponding to each of the test agents is used as the new network security penetration testing strategy for each of the test agents. The steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents are returned until the preset network security penetration testing termination conditions are met.

[0060] The aforementioned network security penetration testing method, apparatus, computer equipment, computer-readable storage medium, and computer program product for power systems, through the observation of the operating status of the power system by each testing agent during the network security penetration testing process, can dynamically update the network security penetration testing strategy corresponding to the testing agent based on a reward function. Therefore, compared with network security penetration testing based on human experience to specify the testing strategy, the network security penetration testing method for power systems based on the above process can dynamically adjust the testing strategy in response to real-time changes in the power system, thereby effectively evaluating the network security performance of the power system. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating a network security penetration testing method for a power system in one embodiment;

[0063] Figure 2 This is a flowchart illustrating the steps of dynamically updating the network security penetration testing strategy executed by each test agent based on a preset reward function and the observation information of each test agent in one embodiment.

[0064] Figure 3 This is a structural block diagram of a cybersecurity penetration testing device for a power system in one embodiment;

[0065] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0068] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0069] In one exemplary embodiment, such as Figure 1As shown, a network security penetration testing method for a power system is provided. This embodiment illustrates the method by applying it to a server. It is understood that this method can also be applied to terminals, and to systems including servers and terminals, and is implemented through interaction between the server and terminals. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. In this embodiment, the method includes the following steps S102 to S110:

[0070] Step S102: Based on the network security penetration test instructions for the power system, determine the network security penetration test strategy for each test agent.

[0071] Different test agents have different roles and tasks in cybersecurity penetration testing. Based on their roles and tasks, different test agents have different action spaces and operating permissions in the power system.

[0072] Among them, network security penetration testing strategies are used to characterize the corresponding testing tasks, testing paths, and testing methods.

[0073] Specifically, when a user needs to conduct cybersecurity penetration testing on the power system's power network, they send a cybersecurity penetration testing command to the server through an interactive interface. The server responds to the command and determines the cybersecurity penetration testing strategy for each testing agent. Here, the power system's power network refers to the power system's computer network.

[0074] In practical applications, after receiving a network security penetration test command, the server first performs preliminary syntax and semantic parsing to analyze whether the user-inputted command is complete, accurate, and executable, and whether it conforms to predefined input format requirements. Then, the server performs power industry-specific verification on the command to determine the testing focus, which may include the test target, scope, and content. Finally, the server optimizes the user-input command using a finely tuned large language model tailored for the power industry, obtaining the target network security penetration test command. Based on this command, the server breaks down the network security penetration test into a series of specific and detailed test tasks, determining the test path and methods for each task, and ultimately generating network security penetration test strategies for each testing agent.

[0075] Step S104: Conduct network security penetration testing on the power system by executing the corresponding network security penetration testing strategies by each testing agent.

[0076] Each test agent is responsible for executing the corresponding cybersecurity penetration testing strategy.

[0077] Specifically, the server performs network security penetration testing on the power system by having each testing agent execute its respective network security penetration testing strategy.

[0078] Step S106: During the network security penetration test, obtain the observation information of each test agent regarding the power system.

[0079] Each test agent is used to observe the operating status of the power system in real time during network security penetration testing, obtaining observational information about the power system. In specific applications, since the operating status of the power system is partially observable, the test agent cannot directly obtain the actual operating status of the power system. Therefore, the test agent uses a Partially Observable Markov Decision Process (POMDP) ​​to construct belief states to represent the probability distribution estimate of the actual operating status of the power system.

[0080] Specifically, each test agent reports its observations to the server; the server receives the observations reported by each test agent.

[0081] Step S108: Based on the preset reward function and the observation information of each test agent, dynamically update the network security penetration testing strategy corresponding to each test agent to obtain the updated network security penetration testing strategy corresponding to each test agent.

[0082] Specifically, for each test moment in the network security penetration test process, the server substitutes the observation information observed by each test agent at the current test moment into a preset reward function, calculates the reward function value for the current test moment, and dynamically updates the network security penetration test strategy corresponding to each test agent at the current test moment based on the reward function value for the current test moment, thereby obtaining the updated network security penetration test strategy corresponding to each test agent, and uses it as the network security penetration test strategy corresponding to each test agent in the next test moment.

[0083] Step S110: The updated network security penetration testing strategy corresponding to each test agent is used as the new network security penetration testing strategy for each test agent. The steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each test agent are returned until the preset network security penetration testing termination conditions are met.

[0084] Each test agent is responsible for executing its corresponding cybersecurity penetration testing strategy at the next test moment.

[0085] Specifically, the server performs continuous network security penetration testing on the power system based on the execution of the network security penetration testing strategy corresponding to each test agent at each test moment, until the preset network security penetration testing termination conditions are met, thus ending the network security penetration testing on the power system. For example, when the test duration of the network security penetration test reaches the preset test duration threshold, or when the system risk value of the power system under the network security penetration test is greater than or equal to the preset system risk threshold.

[0086] In related technologies, testing strategies are typically developed based on human experience, and cybersecurity penetration testing is then conducted on the target based on these strategies. However, for power systems, the topology of the power network and the operating status of the included power equipment change in real time. Cybersecurity penetration testing based on human experience cannot dynamically perceive the overall security posture of the power network. It is prone to triggering global alarms or service interruptions on security devices due to local attacks, and sometimes it may even miss critical vulnerabilities or trigger unexpected risks, making it difficult to effectively assess the cybersecurity performance of the power system.

[0087] In the aforementioned cybersecurity penetration testing method for power systems, the server dynamically updates the cybersecurity penetration testing strategy corresponding to each testing agent based on a reward function by observing the operating status of the power system during the cybersecurity penetration testing process. Therefore, compared with cybersecurity penetration testing based on human experience to specify testing strategies, the cybersecurity penetration testing method for power systems based on the above process can dynamically adjust the testing strategy in response to real-time changes in the power system, thereby effectively evaluating the cybersecurity performance of the power system.

[0088] In an exemplary embodiment, based on the sharing of observation information, each testing agent can also fine-tune its corresponding network security penetration testing strategy; that is, in this application, in addition to the server being able to dynamically update the network security penetration testing strategy of each testing agent as a whole, each testing agent can also make minor adjustments to its own network security penetration testing strategy.

[0089] Specifically, when a test agent observes new information or shares observations with other test agents, it will reassess its belief state based on POMDP and then fine-tune its cybersecurity penetration testing strategy. For example, if a test agent discovers a new vulnerability risk in an area that was previously considered safe through observation or sharing, it will adjust its strategy accordingly, increasing the probability of probing or attacking that area.

[0090] In this embodiment, the test agents can learn from each other by sharing observation information, thereby avoiding duplication of work and conflicts and improving overall collaborative efficiency.

[0091] In one exemplary embodiment, the test agent includes at least a boundary detection agent; the boundary detection agent is positioned as a detector of the "first line of defense" of the power network, responsible for the initial reconnaissance of the boundary area between the external network and the internal network.

[0092] The cybersecurity penetration testing method for power systems provided in this application further includes the following steps:

[0093] By using a boundary detection agent, the power system's power network is subjected to boundary detection. This involves non-intrusive scanning of each boundary device in the power system's power network to determine the device risk value of each boundary device. The boundary detection agent then identifies target boundary devices whose corresponding device risk values ​​are greater than or equal to the device risk threshold. Finally, based on the device risk values ​​of each boundary device, the boundary protection strength assessment result of the power network is determined.

[0094] Specifically, the boundary detection agent is used to perform boundary detection on the power network of the power system. The boundary detection process is as follows: a non-intrusive scan is performed on each boundary device of the power network to determine the device risk value of each boundary device. Among the boundary devices, the boundary devices with device risk values ​​greater than or equal to the preset device risk threshold are identified as target boundary devices. Based on the device risk values ​​of each boundary device, the boundary protection strength of the power network is evaluated, and the boundary protection strength evaluation result of the power network is determined. The target boundary devices and the boundary protection strength evaluation result are reported to the server.

[0095] In practical applications, the boundary detection agent is responsible for non-intrusive scanning of power-specific boundary devices (such as firewalls, routers, vertical encryption authentication gateways, etc.) (e.g., port liveness detection, service fingerprinting, etc.) to mark open high-risk ports and identify configuration weaknesses of boundary devices (e.g., default passwords, weak encryption algorithms), thereby assessing the strength of boundary protection. The boundary detection agent transmits the above-mentioned boundary vulnerability intelligence to the server and other testing agents to provide entry point suggestions for subsequent internal network penetration.

[0096] In this embodiment, through the boundary detection agent, the server can conduct preliminary reconnaissance of the boundary area between the external network and the internal network, thereby providing entry point suggestions for subsequent internal network penetration.

[0097] In one exemplary embodiment, the test agent includes at least a protocol-specific agent; the protocol-specific agent is positioned as a deep analysis expert of the power-specific communication protocol, focusing on vulnerability discovery at the protocol layer.

[0098] The network security penetration testing method for power systems provided in this application further includes the following steps: capturing and parsing communication messages within the power network of the power system using a protocol-specific intelligent agent to determine the corresponding communication protocol of the power network; and determining the design flaws and vulnerabilities of the communication protocol using the protocol-specific intelligent agent.

[0099] Specifically, the protocol-specific intelligent agent is used to capture and reverse analyze communication messages in the power network to determine the communication protocol used by the power network; then, by identifying design flaws and detecting protocol-level vulnerabilities in the communication protocol, the design flaws and vulnerabilities of the communication protocol are determined.

[0100] In practical applications, the protocol-specific intelligent agent is responsible for capturing and reverse-analyzing the messages of the power protocols running in the power network, identifying protocol design defects (such as unencrypted messages or abuse of function codes), and performing protocol-level vulnerability detection to verify whether there are exploitable logical vulnerabilities (such as tampering with protection settings or forging meter data). Then, the protocol-specific intelligent agent generates a protocol vulnerability report and provides protocol interaction parameters (such as message field constraints and legal value ranges) for the vulnerabilities.

[0101] In this embodiment, through a protocol-specific intelligent agent, the server is able to perform protocol-layer vulnerability mining on the power-specific communication protocol of the power network.

[0102] In one exemplary embodiment, the test agent includes at least an intranet penetration agent; the role of the intranet penetration agent is that of a lateral movement attacker in the intranet environment, responsible for deep penetration after breaching the boundary.

[0103] The network security penetration testing method for power systems provided in this application further includes the following steps: using an intranet penetration agent to perform lateral movement attacks on critical target intranet devices in the power network of the power system; and using the intranet penetration agent to perform lateral movement attacks on the target intranet devices to obtain sensitive data of the power system from the target intranet devices.

[0104] Among them, the target intranet devices are those intranet devices in the power network that are more or less important to the power system than the importance value.

[0105] Lateral movement attacks, in particular, are a series of attack behaviors employed by attackers after breaching the initial boundaries of a network. These attacks involve expanding the attacker's control and accessing core assets across devices and regions within the internal network environment. Essentially, it represents a crucial step in the attacker's transition from a "single-point breach" to "internal network penetration." The core objective is to gain higher privileges, control more critical systems, and ultimately achieve the goals of stealing data, disrupting business operations, or establishing a long-term presence within the network.

[0106] Specifically, the intranet penetration agent is used to launch lateral movement attacks on key equipment in the power network (such as SCADA servers, historical databases, and relay protection devices) based on the entry points provided by the boundary detection agent, in order to improve data access privileges and obtain sensitive data of the power system from the key equipment.

[0107] In this embodiment, through an intranet penetration agent, the server can perform lateral movement attacks on key equipment in the power network to obtain sensitive data.

[0108] In one exemplary embodiment, the test agent includes at least a socially engineered agent; the role of the socially engineered agent is that of an attacker targeting “human weaknesses” of the power system’s workers.

[0109] The cybersecurity penetration testing method for power systems provided in this application further includes the following steps: conducting simulated phishing attacks on the power system using social engineering agents; and obtaining target operation logs within the power system using social engineering agents.

[0110] Among them, the target operation log is the operation log in each operation log in the power system used to record human operations in response to simulated phishing attacks.

[0111] Simulated phishing attacks are a type of security testing activity initiated by enterprises or professional security teams under legal authorization to simulate real phishing attacks.

[0112] Specifically, a social engineering agent is used to simulate phishing attacks on a power system, and after the simulated phishing attack, to obtain target operation logs within the power system used to record the actions taken by power system staff in response to the simulated phishing attack.

[0113] In practical applications, social engineering agents are responsible for collecting the operation logs of attacked personnel, evaluating the effectiveness of security training for power system staff, and assisting in adjusting the overall penetration strategy.

[0114] In this embodiment, through a social engineering agent, the server can assess the effectiveness of safety training for power system personnel, thereby assisting in adjusting the overall penetration strategy.

[0115] In one exemplary embodiment, the testing agent includes at least an emergency assessment agent; the emergency assessment agent is positioned as a "security gatekeeper" of the penetration testing process, used to monitor the impact of the penetration test on the operation of the power system in real time.

[0116] The cybersecurity penetration testing method for power systems provided in this application further includes the following steps: using an emergency assessment agent to monitor the operating status of the power system in real time, capturing alarm information in the power system and determining whether there are emergency response actions for the alarm information in the power system; and ending the cybersecurity penetration test of the power system when the emergency assessment agent determines, based on the operating status of the power system, that the system risk value of the power system is greater than or equal to the system risk threshold.

[0117] Specifically, the emergency assessment agent is used to monitor the operating status of the power system in real time, capture alarm information in the power system, and determine whether there are emergency response actions in response to the alarm information, such as whether the emergency response process of the power grid company has been triggered. When the system risk value of the power system is detected to be greater than or equal to the preset system risk threshold based on the operating status of the power system, such as detecting high-risk impacts such as substation communication interruption or real-time control command tampering, a termination signal is immediately sent to the server. The server responds to the termination signal and ends the network security penetration test of the power system.

[0118] In practical applications, the emergency assessment agent is responsible for monitoring the real-time status of the power system (such as the delay of control commands in the SCADA system and the frequency of meter data uploads), determining whether anomalies are caused by penetration testing (such as malfunction of protection devices or interruption of data acquisition), and simultaneously capturing alarm information from power safety devices in the power system and assessing whether the emergency response process of the power grid company has been triggered. When a high-risk impact is detected (such as substation communication interruption or tampering with real-time control commands), the emergency assessment agent immediately sends a termination signal to the server to suspend all attack actions.

[0119] In this embodiment, through an emergency assessment agent, the server can monitor the impact of penetration testing on the operation of the power system in real time, so as to ensure that penetration testing is carried out without jeopardizing the safe and stable operation of the power system.

[0120] In an exemplary embodiment, the reward function includes a first reward function, a second reward function, and a third reward function; wherein, the first reward function is used to quantify the value of each test agent's attack actions on the power system for security penetration testing; the second reward function is used to constrain the impact of network security penetration testing on the stable operation of the power system; and the third reward function is used to optimize the collaborative efficiency of each test agent.

[0121] like Figure 2 As shown, step S108 above, based on a preset reward function and the observation information of each test agent, dynamically updates the corresponding network security penetration testing strategy executed by each test agent, specifically including the following steps:

[0122] Step S202: Based on the observation information of each test agent, determine the first function value of the network security penetration test under the first reward function, the second function value under the second reward function, and the third function value under the third reward function.

[0123] Step S204: Weighted fusion of the first function value, the second function value, and the third function value to obtain the current total reward value for the network security penetration test.

[0124] Step S206: Dynamically update the cybersecurity penetration testing strategy corresponding to each testing agent based on the total reward value.

[0125] Specifically, the server pre-builds the following total reward function:

[0126] (Formula 1)

[0127] in, For a moment ; The preset total reward function; The first reward function is preset. The first weight corresponds to the first reward function; For the preset second reward function, This refers to the second weight corresponding to the second reward function; For the preset third reward function, This is the third weight corresponding to the third reward function.

[0128] The first reward function is shown in Formula 2:

[0129] (Formula 2)

[0130] in, The first discovered for the attack action One vulnerability; For the first The impact factors of each vulnerability; For the first The score of the first vulnerability in CCVS (Common Vulnerability Scoring System) is used to quantify the vulnerability's score. The severity of each vulnerability; The first The old and new privilege levels of the vulnerability; All are constant parameters.

[0131] In practical applications, the server can also add reward items corresponding to privilege escalation to the first reward function. For example, the higher the degree of privilege escalation brought about by the attack action and the higher the privilege level escalated, the greater the reward will be.

[0132] The second reward function is shown in Formula 3:

[0133] (Formula 3)

[0134] in, These are the penalty coefficients for frequency and voltage, respectively; These are frequency deviation and voltage deviation, respectively. This is the tolerance threshold for voltage deviation.

[0135] The third reward function is shown in Formula 4:

[0136] (Formula 4)

[0137] in, The total number of test agents; For the first to participate in the collaboration One test agent; For the first A test agent at time... A comprehensive score for each instance of contribution to cybersecurity penetration testing; The length of the sliding time window; This is the time decay factor; For historical time step variables; For the set of all test agents; Not including the first Any subset of test agents; To test a subset of intelligent agents At any moment The value created by time; For the first A test agent at time... Marginal contribution to the team at that time.

[0138] In practical applications, the server calculates the first function value of the network security penetration test under the preset first reward function based on Formula 2, the second function value of the network security penetration test under the preset second reward function based on Formula 3, and the third function value of the network security penetration test under the preset third reward function based on Formula 4. Then, based on Formula 1, the first function value, the second function value, and the third function value are weighted and fused to obtain the current total reward value of the network security penetration test. Finally, the server dynamically updates the network security penetration test strategy corresponding to each test agent based on the total reward value.

[0139] In this embodiment, through environmental state perception and multi-dimensional coupled computation, the server achieves precise guidance of the test agent's behavior. The aforementioned reward function breaks through the single-dimensional reward mechanism in traditional penetration testing, creatively constructing a "three-dimensional dynamic evaluation system." It deeply integrates the physical characteristics of the power system, the laws of network security attack and defense, and the principle of multi-agent collaboration, realizing a paradigm upgrade from "static scalar reward" to "dynamic tensor evaluation." In specific implementation, the preset reward function adopts a three-layer architecture design: the first reward function is responsible for evaluating the value of basic attack actions such as vulnerability discovery and privilege escalation to accurately quantify technical achievements; the second reward function focuses on power system stability maintenance, using an exponential penalty function based on real-time data from the PMU (Phasor Measurement Unit) and a voltage transient compensation mechanism to ensure that penetration testing does not affect the normal operation of the power grid; the third reward function, through the Shapley value algorithm (a classic method in game theory used to solve the problem of fair distribution of benefits (or costs) in cooperative games) and time-series window optimization, realizes the task allocation and contribution measurement among multiple agents, significantly improving team collaboration efficiency.

[0140] To more clearly illustrate the cybersecurity penetration testing method for power systems provided in this application, a specific embodiment is used below to describe the method. However, it should be understood that the embodiments of this application are not limited thereto. In one exemplary embodiment, this application also provides a cybersecurity penetration testing method for power systems based on a POMDP multi-agent cooperative system, specifically including the following:

[0141] I. Test System Architecture.

[0142] This testing system includes a central control system and multiple intelligent agents; the multiple intelligent agents include at least a boundary detection intelligent agent, a protocol-specific intelligent agent, an intranet penetration intelligent agent, a social engineering intelligent agent, and an emergency assessment intelligent agent.

[0143] II. Training and Continuous Learning of the Testing System.

[0144] Data Collection and Preprocessing: Collect a large amount of real-world cybersecurity penetration test data, including network traffic data, vulnerability exploitation examples, and security incident records. Perform preprocessing operations such as cleaning, labeling, and feature extraction on this data to construct a dataset suitable for model training.

[0145] Application of Deep Reinforcement Learning Algorithms: The POMDP (Programmatic Multi-Agent Collaborative Learning) deep reinforcement learning algorithm is used for training. The agents learn optimal action strategies under different states and observations through extensive interaction with the simulated network environment. During training, the parameters of the neural network are continuously adjusted to optimize the expected value of the reward function, thereby improving the model's performance and generalization ability.

[0146] Continuous Learning and Update Mechanism: To address the ever-emerging new vulnerabilities, attack methods, and defense technologies in the cybersecurity field, a continuous learning and update mechanism has been established. This mechanism regularly learns from the latest comprehensive cybersecurity knowledge graph and penetration testing response specification vector database, updating and optimizing its own strategies to maintain their effectiveness and adaptability.

[0147] III. Penetration testing of the testing system.

[0148] The central control system is used to decompose the penetration test task into multiple sub-tasks based on the complexity of the power network to be tested, the capabilities and characteristics of the agents, and the current penetration test progress. This is done by employing a dynamic task allocation strategy and assigning them to different agents. This ensures that the work of each agent is coordinated and does not conflict with each other, thereby improving the overall penetration test efficiency.

[0149] Multiple intelligent agents share observed network information, vulnerability data, as well as their own status and plans in real time, so that other intelligent agents can adjust their behavior strategies in a timely manner based on this information and achieve collaborative operations.

[0150] Each agent collaborates on reasoning and decision-making based on the POMDP model and shared information, jointly determining the optimal penetration testing path and strategy by analyzing each other's belief states and action plans.

[0151] The central control system achieves precise guidance of agent behavior through environmental state perception and multi-dimensional coupled computation. Specifically, it creatively constructs a "three-dimensional dynamic evaluation system": the first reward function is responsible for evaluating the value of basic attack actions such as vulnerability discovery and privilege escalation to accurately quantify technical achievements; the second reward function focuses on power system stability maintenance, employing an exponential penalty function based on PMU real-time data and a voltage transient compensation mechanism to ensure that penetration testing does not affect the normal operation of the power grid; the third reward function, through the Shapley value algorithm and time-series window optimization, realizes task allocation and contribution measurement among multiple agents, significantly improving team collaboration efficiency. These reward functions deeply integrate the physical characteristics of the power system, the laws of network security attack and defense, and the principles of multi-agent collaboration, achieving a paradigm upgrade from "static scalar rewards" to "dynamic tensor evaluation."

[0152] IV. Test system output.

[0153] The final output is a multi-dimensional penetration test report, including vulnerability detection reports and abnormal access detection reports.

[0154] In this embodiment, firstly, it has higher adaptability to the power industry, improving the vulnerability discovery rate and reducing the false positive rate; secondly, it has higher adaptability to dynamic environments, improving test coverage and reducing the risk of business interruption through POMDP-based adaptive adjustment; thirdly, it has higher multi-agent collaboration efficiency, improving testing efficiency and overall penetration success rate through dynamic task allocation strategies and collaborative reasoning mechanisms; fourthly, it balances security and testing effectiveness, ensuring power grid security and enhancing testing effectiveness through reward functions; and fifthly, through continuous system learning, it can adapt to new equipment, new protocol versions, and new attack methods constantly emerging in the power sector without frequent manual retraining.

[0155] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0156] Based on the same inventive concept, this application also provides a network security penetration testing device for a power system to implement the aforementioned network security penetration testing method for a power system. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations in one or more embodiments of the network security penetration testing device for a power system provided below can be found in the limitations of the network security penetration testing method for a power system described above, and will not be repeated here.

[0157] In one exemplary embodiment, such as Figure 3 As shown, a network security penetration testing device for a power system is provided, comprising: a policy determination module 302, a penetration testing module 304, an information observation module 306, and a policy update module 308, wherein:

[0158] The strategy determination module 302 is used to determine the network security penetration testing strategy for each testing agent based on the network security penetration testing instructions for the power system.

[0159] The penetration testing module 304 is used to perform network security penetration testing on the power system by executing the corresponding network security penetration testing strategies through each testing agent.

[0160] The information observation module 306 is used to acquire observation information of the power system by each test agent during the network security penetration test; the observation information is obtained by the test agents observing the operating status of the power system.

[0161] The strategy update module 308 is used to dynamically update the network security penetration testing strategy corresponding to each test agent based on the preset reward function and the observation information of each test agent, so as to obtain the updated network security penetration testing strategy corresponding to each test agent.

[0162] The penetration testing module 304 is also used to take the updated network security penetration testing strategy corresponding to each test agent as the new network security penetration testing strategy for each test agent, and return the steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each test agent until the preset network security penetration testing termination conditions are met.

[0163] In one exemplary embodiment, the test agent includes at least a boundary detection agent.

[0164] The penetration testing module 304 is also used to perform non-intrusive scanning of each boundary device of the power network of the power system through a boundary detection agent to determine the device risk value of each boundary device; to identify the target boundary device with a corresponding device risk value greater than or equal to the device risk threshold among each boundary device through the boundary detection agent; and to determine the boundary protection strength assessment result of the power network based on the device risk value of each boundary device.

[0165] In one exemplary embodiment, the test agent includes at least a protocol-specific agent.

[0166] The penetration testing module 304 is also used to capture and parse communication messages within the power network of the power system through a protocol-specific intelligent agent to determine the communication protocol corresponding to the power network; and to determine the design flaws and vulnerabilities of the communication protocol through the protocol-specific intelligent agent.

[0167] In one exemplary embodiment, the test agent includes at least an intranet penetration agent.

[0168] The penetration testing module 304 is also used to perform lateral movement attacks on target internal network devices in the power network of the power system through an internal network penetration agent; the target internal network devices are internal network devices in the power network whose importance to the power system is greater than or equal to the importance value; and sensitive data of the power system is obtained from the target internal network devices through the lateral movement attack of the internal network penetration agent.

[0169] In one exemplary embodiment, the test agent includes at least a socially engineered agent.

[0170] The penetration testing module 304 is also used to simulate phishing attacks on the power system through social engineering agents; and to obtain target operation logs within the power system through social engineering agents; the target operation logs are operation logs from various operation logs within the power system that record human operations in response to the simulated phishing attacks.

[0171] In one exemplary embodiment, the test agent includes at least an emergency assessment agent.

[0172] The penetration testing module 304 is also used to monitor the operating status of the power system in real time through the emergency assessment agent, capture alarm information in the power system and determine whether there are emergency response actions for the alarm information in the power system; if the emergency assessment agent determines that the system risk value of the power system is greater than or equal to the system risk threshold based on the operating status of the power system, the network security penetration test of the power system will end.

[0173] In an exemplary embodiment, the policy update module 308 is further configured to determine, based on the observation information of each test agent, a first function value of the network security penetration test under a preset first reward function, a function value under a preset second reward function, and a third function value under a preset third reward function; weightedly fuse the first function value, the second function value, and the third function value to obtain the current total reward value of the network security penetration test; and dynamically update the network security penetration test policy corresponding to each test agent according to the total reward value; wherein, the first reward function is used to quantify the value of each test agent's attack actions on the power system to the security penetration test; the second reward function is used to constrain the impact of the network security penetration test on the stable operation of the power system; and the third reward function is used to optimize the collaborative efficiency of each test agent.

[0174] The modules in the aforementioned cybersecurity penetration testing device for power systems can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0175] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a network security penetration testing method for a power system.

[0176] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0177] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0178] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0179] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0180] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0181] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0182] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A network security penetration testing method for a power system, characterized in that, The method includes: Based on the cybersecurity penetration test instructions for the power system, determine the cybersecurity penetration test strategies for each test agent. By executing the corresponding network security penetration testing strategy by each of the aforementioned test agents, network security penetration testing is performed on the power system. During the network security penetration test, observation information of each test agent regarding the power system is acquired; the observation information is obtained through the test agents' observation of the power system's operating status. Based on a preset reward function and the observation information of each test agent, the network security penetration testing strategy corresponding to each test agent is dynamically updated to obtain the updated network security penetration testing strategy corresponding to each test agent; further comprising: based on the observation information of each test agent, determining a first function value of the network security penetration test under a first reward function, a second function value under a second reward function, and a third function value under a third reward function; the first reward function is used to quantify the value of each test agent's attack actions on the power system to the network security penetration test; the second reward function is used to constrain the impact of the network security penetration test on the stable operation of the power system; the third reward function is used to optimize the collaboration efficiency of each test agent; the first function value, the second function value, and the third function value are weighted and fused to obtain the current total reward value of the network security penetration test; and the network security penetration testing strategy corresponding to each test agent is dynamically updated according to the total reward value. The updated network security penetration testing strategy corresponding to each of the test agents is used as the new network security penetration testing strategy for each of the test agents. The steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents are returned until the preset network security penetration testing termination conditions are met.

2. The method according to claim 1, characterized in that, The test agent includes at least a boundary detection agent; The method further includes: The boundary detection agent performs a non-intrusive scan of each boundary device of the power system's power network to determine the device risk value of each boundary device. The boundary detection agent identifies target boundary devices whose risk values ​​are greater than or equal to the risk threshold among the boundary devices, and determines the boundary protection strength assessment result of the power network based on the risk values ​​of each boundary device.

3. The method according to claim 1, characterized in that, The test agent includes at least a protocol-specific agent; The method further includes: The protocol-specific intelligent agent captures and parses communication messages within the power network of the power system to determine the communication protocol corresponding to the power network. The design flaws and vulnerabilities of the communication protocol are identified through the protocol-specific intelligent agent.

4. The method according to claim 1, characterized in that, The test agent includes at least an intranet penetration agent; The method further includes: The intranet penetration agent performs lateral movement attacks on target intranet devices in the power system's power network; the target intranet devices are those whose importance to the power system is greater than or equal to the importance value among the various intranet devices in the power network. Sensitive data about the power system is obtained from the target intranet device through the lateral movement attack of the intranet penetration agent.

5. The method according to claim 1, characterized in that, The test agents include at least social engineering agents; The method further includes: The social engineering agent was used to conduct a simulated phishing attack on the power system. The social engineering agent obtains the target operation logs within the power system; the target operation logs are the operation logs among the various operation logs within the power system used to record human operations in response to the simulated phishing attack.

6. The method according to claim 1, characterized in that, The test agent includes at least an emergency assessment agent; The method further includes: The emergency assessment agent monitors the operating status of the power system in real time, captures alarm information in the power system, and determines whether there is an emergency response action in response to the alarm information in the power system. If the emergency assessment agent determines, based on the operating status of the power system, that the system risk value of the power system is greater than or equal to the system risk threshold, the cybersecurity penetration test on the power system will be terminated.

7. A security penetration testing device for a power system, characterized in that, The device includes: The strategy determination module is used to determine the network security penetration testing strategy for each testing agent based on the network security penetration testing instructions for the power system. The penetration testing module is used to perform network security penetration testing on the power system by executing the corresponding network security penetration testing strategies through each of the test agents. An information observation module is used to acquire observation information of the power system by each test agent during the network security penetration test; the observation information is obtained by the test agents observing the operating status of the power system. The strategy update module is used to dynamically update the network security penetration testing strategy corresponding to each of the test agents based on a preset reward function and the observation information of each of the test agents, so as to obtain the updated network security penetration testing strategy corresponding to each of the test agents. The penetration testing module is also used to take the updated network security penetration testing strategy corresponding to each of the test agents as the new network security penetration testing strategy for each of the test agents, and return the steps of performing network security penetration testing on the power system by executing the corresponding network security penetration testing strategy by each of the test agents until the preset network security penetration testing termination conditions are met. The strategy update module is further configured to determine, based on the observation information of each of the test agents, a first function value under a first reward function, a second function value under a second reward function, and a third function value under a third reward function for the network security penetration test; the first reward function is used to quantify the value of each test agent's attack actions on the power system to the network security penetration test; the second reward function is used to constrain the impact of the network security penetration test on the stable operation of the power system; the third reward function is used to optimize the collaboration efficiency of each of the test agents; the first function value, the second function value, and the third function value are weighted and fused to obtain the current total reward value of the network security penetration test; and the network security penetration test strategy corresponding to each test agent is dynamically updated according to the total reward value.

8. The apparatus according to claim 7, characterized in that, The test agent includes at least a boundary detection agent; The penetration testing module is also used to perform non-intrusive scanning of each boundary device of the power network of the power system through the boundary detection agent, and determine the device risk value of each boundary device; The boundary detection agent identifies target boundary devices whose risk values ​​are greater than or equal to the risk threshold among the boundary devices, and determines the boundary protection strength assessment result of the power network based on the risk values ​​of each boundary device.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images