Automated penetration testing methods, apparatus, equipment and storage media

Abstract

This invention discloses an automated penetration testing method, apparatus, device, and storage medium, belonging to the field of penetration testing technology. The invention obtains initial information about a real network environment; inputs this initial information into a preset automated penetration testing agent to obtain target attack actions; loads the target attack actions into the target network environment and receives feedback information from the target network environment; and performs automated penetration testing based on the feedback information. This solution for automated penetration testing does not excessively rely on human resources, enabling automated execution and fine-grained control of the entire sequential penetration testing process, improving the accuracy and adaptability of the test, and thus more effectively identifying potential security vulnerabilities and threats.

Description

Technical Field

[0001] This invention relates to the field of penetration testing technology, and in particular to an automated penetration testing method, apparatus, equipment, and storage medium. Background Technology

[0002] With the rapid evolution of technologies such as the Internet, big data, and the Internet of Things, an increasing number of devices are connected to the network. However, numerous potential security vulnerabilities exist, and the lack of effective protection measures makes these devices potential targets for malicious attackers. To improve cybersecurity, security professionals have proposed defense-oriented automated attack techniques, often referred to as automated penetration testing.

[0003] Automated penetration testing represents a methodology for discovering and exploiting potential vulnerabilities in computer systems. This technique enables automated security testing and risk assessment of target systems, simulating an attacker's actions, thereby helping to address or mitigate potential security vulnerabilities and risks. This technology can detect potential security risks in a timely manner and facilitates the remediation of vulnerabilities before cyberattacks occur, improving the overall security of the network.

[0004] Automated penetration testing technology offers multiple benefits: it effectively assists security experts in detecting and locating potential security vulnerabilities and risks in network systems, providing relevant advice and solutions. Simultaneously, it reduces the workload of traditional manual penetration testing, lowers human resource costs, and improves testing efficiency.

[0005] Existing automated penetration testing tools sometimes generate false alarms, misidentifying normal configurations or operations as vulnerabilities. On the other hand, they may also miss genuine vulnerabilities because they cannot fully understand the complexity of the application, resulting in low accuracy in automated penetration testing. Summary of the Invention

[0006] The main objective of this invention is to provide an automated penetration testing method, apparatus, device, and storage medium, aiming to solve the technical problem of low accuracy in existing penetration testing technologies.

[0007] To achieve the above objectives, the present invention provides an automated penetration testing method, the method comprising the following steps:

[0008] Obtain initial information about the real network environment;

[0009] The initial information of the real network environment is input into a preset automated penetration testing agent to obtain the target attack action;

[0010] The target attack action is loaded into the target network environment, and feedback information from the target network environment is received.

[0011] Automated penetration testing is performed based on the feedback information.

[0012] Optionally, the step of inputting the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action includes:

[0013] Based on the initial information of the real network environment, obtain the real network environment information and the corresponding sub-action mask information;

[0014] The real network environment information and the sub-action mask information are input into a preset automated penetration testing agent to obtain the target attack action.

[0015] Optionally, the step of inputting the real network environment information and the sub-action mask information into a preset automated penetration testing agent to obtain the target attack action includes:

[0016] The real network environment information and the sub-action mask information are input into a preset automated penetration testing agent. The preset automated penetration testing agent processes the real network environment information and outputs attack sub-actions of various dimensions.

[0017] The attack sub-actions of each dimension are processed by the sub-action mask information to obtain the preset probability distribution of each attack sub-action.

[0018] The target attack sub-action is obtained by using random decision-making through the preset probability distribution to filter the attack sub-actions of each dimension.

[0019] The target attack sub-actions are assembled to obtain the target attack action.

[0020] Optionally, the automated penetration testing based on the feedback information includes:

[0021] Based on the feedback information, the updated network environment information and the corresponding update sub-action mask are obtained;

[0022] The updated network environment information and the updated sub-action mask are used as the initial information for updating the real network environment. The process of inputting the initial information of the real network environment into the preset automated penetration testing agent to obtain the target attack action continues until the attack target or number of attacks in the target network environment meets the preset number of attacks.

[0023] Optionally, before inputting the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action, the method further includes:

[0024] Collect penetration test sample log data;

[0025] The initial automated penetration testing agent is trained using the penetration test sample log data to construct a preset automated penetration testing agent.

[0026] Optionally, the collection of penetration test sample log data includes:

[0027] Collect real attack data and initial attack results;

[0028] Obtain the attack strategy and second attack results of the online automated penetration testing agent;

[0029] Construct a network environment and launch an attack using a preset attack strategy within the network environment, recording sample attack actions and third-party attack results;

[0030] Test sample log data is obtained by using the real attack data, the first attack result, the attack strategy, the second attack result, the sample attack action, and the third attack result.

[0031] Optionally, training the initial automated penetration testing agent using the penetration testing sample log data to construct a preset automated penetration testing agent includes:

[0032] Based on the penetration test sample log data, a preset reinforcement learning strategy is used to train the initial automated penetration test agent to obtain a reference automated penetration test agent.

[0033] The penetration testing performance and efficiency of the reference automated penetration testing agent were tested in a real network environment, and the test results were obtained.

[0034] Based on the test results, the parameters of the reference automated penetration testing agent are adjusted to construct a preset automated penetration testing agent.

[0035] Furthermore, to achieve the above objectives, the present invention also proposes an automated penetration testing device, the automated penetration testing device comprising:

[0036] The acquisition module is used to acquire initial information about the real network environment;

[0037] The input module is used to input the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action;

[0038] The loading module is used to load the target attack action into the target network environment and receive feedback information from the target network environment.

[0039] The testing module is used to perform automated penetration testing based on the feedback information.

[0040] Furthermore, to achieve the above objectives, the present invention also proposes an automated penetration testing device, which includes: a memory, a processor, and an automated penetration testing program stored in the memory and executable on the processor, wherein the automated penetration testing program is configured to implement the steps of the automated penetration testing method described above.

[0041] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing an automated penetration testing program, which, when executed by a processor, implements the steps of the automated penetration testing method described above.

[0042] This invention obtains initial information about a real network environment; inputs this initial information into a preset automated penetration testing agent to obtain target attack actions; loads the target attack actions into the target network environment and receives feedback information from the target network environment; and performs automated penetration testing based on the feedback information. This solution for automated penetration testing does not overly rely on human resources, enabling automated execution and fine-grained control of the entire sequential penetration testing process, improving the accuracy and adaptability of the test, and thus more effectively identifying potential security vulnerabilities and threats. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the structure of an automated penetration testing device for the hardware operating environment involved in the embodiments of the present invention;

[0044] Figure 2 This is a flowchart illustrating the first embodiment of the automated penetration testing method of the present invention;

[0045] Figure 3 This is a flowchart illustrating the second embodiment of the automated penetration testing method of the present invention;

[0046] Figure 4 This is a schematic diagram of the architecture of a preset automated penetration testing agent in an embodiment of the automated penetration testing method of the present invention;

[0047] Figure 5 This is a flowchart illustrating the third embodiment of the automated penetration testing method of the present invention;

[0048] Figure 6 This is a flowchart illustrating the fourth embodiment of the automated penetration testing method of the present invention;

[0049] Figure 7 This is a schematic diagram of the overall process of automated penetration testing according to an embodiment of the automated penetration testing method of the present invention;

[0050] Figure 8 This is a structural block diagram of the first embodiment of the automated penetration testing device of the present invention.

[0051] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0053] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of an automated penetration testing device for the hardware operating environment involved in the embodiments of the present invention.

[0054] like Figure 1 As shown, the automated penetration testing equipment may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on automated penetration testing equipment and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0056] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an automated penetration testing program.

[0057] exist Figure 1In the automated penetration testing device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the automated penetration testing device of the present invention can be set in the automated penetration testing device, and the automated penetration testing device calls the automated penetration testing program stored in the memory 1005 through the processor 1001 and executes the automated penetration testing method provided in the embodiment of the present invention.

[0058] This invention provides an automated penetration testing method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the automated penetration testing method of the present invention.

[0059] In this embodiment, the automated penetration testing method includes the following steps:

[0060] Step S10: Obtain initial information about the real network environment.

[0061] It should be noted that the execution subject of this embodiment can be an automated penetration testing device, or other devices that can achieve the same or similar functions. This embodiment does not limit this; this embodiment uses an automated penetration testing device as an example for explanation.

[0062] Currently, most automated penetration testing techniques have not fully integrated artificial intelligence, thus lacking the intelligent judgment and decision-making capabilities of human testers. This results in limitations for automated penetration testing tools, including the potential to miss some potential vulnerabilities or security risks, or to provide inaccurate test results. Furthermore, they cannot perform personalized penetration testing based on specific scenarios like human testers. To achieve full automation of penetration testing and introduce intelligently adaptive and resilient penetration testing strategies, automated penetration testing techniques need to possess the following key capabilities:

[0063] It can execute attack decisions coherently, just like a human tester makes continuous decisions based on the complexity and interrelationships of vulnerabilities.

[0064] It has a dynamic learning mechanism to quickly adjust testing strategies and generate new attack methods based on newly discovered vulnerabilities and attack vectors, as well as the constantly changing network environment.

[0065] These improvements will help automated penetration testing technology better simulate the intelligent judgment and decision-making process of human testers, improve the accuracy and adaptability of testing, and thus more effectively identify potential security vulnerabilities and threats.

[0066] To address the aforementioned shortcomings, the automated penetration testing method proposed in this embodiment trains a penetration testing agent based on offline reinforcement learning. This agent is capable of learning real network attack behavior patterns from penetration testing log data. It can quickly make new attack decisions based on observed network environment information, achieving full automation and highly granular attack control throughout the entire penetration testing process. By utilizing an attack arsenal, the offline reinforcement learning model can mimic attacker behavior patterns. It can automatically select the attack target, attack method, and specific attack parameters for each attack, enabling the automatic execution of a sequential attack process, thereby more effectively identifying potential security vulnerabilities and threats.

[0067] It should be noted that the initial information of the real network environment refers to the initial information obtained in the actual network environment. The initial information of the real network environment may include the network environment information observed during initialization and the corresponding sub-action mask.

[0068] Step S20: Input the initial information of the real network environment into the preset automated penetration testing agent to obtain the target attack action.

[0069] In practice, the pre-trained automated penetration testing agent is trained in advance using an offline reinforcement learning algorithm. This pre-trained agent can automatically execute penetration testing tasks and assess the security of the target system by simulating hacker attacks.

[0070] In practice, the pre-set automated penetration testing agent is deployed in the target test network. Therefore, when automated penetration testing is required, the pre-set automated penetration testing agent can be loaded, and the initial information of the real network environment can be input into the pre-set automated penetration testing agent. The pre-set automated penetration testing agent can then perform a series of processes on the real network environment information, such as feature embedding and feature extraction, to obtain the target attack actions.

[0071] The pre-defined automated penetration testing agent's actions are represented as a specific cybersecurity attack carried out in the network.

[0072] Step S30: Load the target attack action into the target network environment and receive feedback information from the target network environment.

[0073] It is understandable that the target attack action can be loaded into the target network environment to carry out a network security attack and receive feedback information from the target network environment regarding this target attack action.

[0074] Step S40: Perform automated penetration testing based on the feedback information.

[0075] In practice, automated penetration testing can be conducted based on feedback information. By continuously receiving feedback information from the target network environment, attack testing can be performed, thereby completing automated penetration testing.

[0076] This embodiment obtains initial information about a real network environment; inputs this initial information into a preset automated penetration testing agent to obtain target attack actions; loads the target attack actions into the target network environment and receives feedback information from the target network environment; and performs automated penetration testing based on the feedback information. This solution for automated penetration testing does not overly rely on human resources, enabling automated execution and fine-grained control of the entire sequential penetration testing process, improving the accuracy and adaptability of the test, and thus more effectively identifying potential security vulnerabilities and threats.

[0077] refer to Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the automated penetration testing method of the present invention.

[0078] Based on the first embodiment described above, step S20 of the automated penetration testing method in this embodiment includes:

[0079] Step S201: Obtain the real network environment information and the corresponding sub-action mask information based on the initial information of the real network environment.

[0080] It should be noted that the initial information of the real network environment includes the real network environment information observed during initialization and the corresponding sub-action mask information. Therefore, the real network environment information and the corresponding sub-action mask information can be obtained from the initial information of the real network environment.

[0081] Step S202: Input the real network environment information and the sub-action mask information into the preset automated penetration testing agent to obtain the target attack action.

[0082] Understandably, the observed real network environment information and sub-action mask information can be input into a preset automated penetration testing agent, which can then process the information to obtain the target attack action.

[0083] The target attack action is a complete attack action assembled from multiple parts. Specifically, the steps of inputting the real network environment information and the sub-action mask information into a preset automated penetration testing agent to obtain the target attack action include: inputting the real network environment information and the sub-action mask information into the preset automated penetration testing agent; processing the real network environment information through the preset automated penetration testing agent to output attack sub-actions of each dimension; processing the attack sub-actions of each dimension through the sub-action mask information to obtain a preset probability distribution of the attack sub-actions of each dimension; using random decision-making based on the preset probability distribution to filter the attack sub-actions of each dimension to obtain the target attack action; and assembling the target attack sub-actions to obtain the target attack action.

[0084] Understandably, real network environment information and sub-action mask information can be input into a preset automated penetration testing agent, which can then process the real network environment information and output attack sub-actions of various dimensions.

[0085] like Figure 4 As shown, Figure 4 This diagram illustrates the architecture of a pre-defined automated penetration testing agent. The agent employs an actor-commentator architecture, comprising an actor deep neural network for outputting the attack strategy distribution and a commentator deep neural network for performing value function estimation. Both the actor and commentator deep neural networks share some neural network parameters. Specifically, the automated penetration testing agent collects observations of the target network (including currently discovered network node information, currently compromised network node information, discovered port information, and discovered leaked key information). The agent first performs feature embedding on the collected network observations, then uses a fully connected deep neural network to further extract features from the embedded information, obtaining the final high-dimensional observation features. The fully connected deep neural network uses the ReLU function as the non-linear activation function. Based on the processed high-dimensional observation features, two different neural network branches are used to process the value function estimation and strategy distribution output respectively, thus achieving the actor-commentator neural network architecture with partially shared neural network parameters.

[0086] The commentator network directly inputs the encoded high-dimensional observation feature information into a new fully connected neural network, and then outputs an observation value estimate.

[0087] The design of actor networks is quite complex and requires careful consideration of the specific characteristics of automated penetration testing scenarios. In automated penetration testing scenarios, a single action of an agent based on offline reinforcement learning represents a specific cybersecurity attack carried out within the network. Cybersecurity attacks exhibit a clear decomposability property, meaning that a single cybersecurity attack can be composed of multiple specific attack sub-actions (including the attack initiating node, the target node, the attack type, and specific attack parameters).

[0088] This solution considers three different types of network security attack forms (local attacks, remote attacks, and lateral movement).

[0089] A local attack needs to consider the attack initiating node, the target node, and the target's local vulnerabilities.

[0090] A remote attack needs to consider the attack initiating node, the target node, and the target remote vulnerability.

[0091] A lateral movement requires consideration of the attack initiating node, the attack target node, the target port, and the key credentials used in the attack.

[0092] In this embodiment, the actor network adopts a multi-branch structure, that is, multiple different deep neural network branches make decisions on different attack sub-action dimensions, and then the output attack sub-actions of each dimension are assembled into a complete network security attack action and injected into the test target network for execution.

[0093] The actor network consists of five attack sub-actions: attack target node sub-action, attack type sub-action, target local vulnerability sub-action, target remote vulnerability sub-action, and target port sub-action. The attack target node and attack type constitute the main attack action, while the target local vulnerability, target remote vulnerability, and target port constitute the attack parameters.

[0094] Similar to the critic network, each sub-action branch of the actor network inputs encoded high-dimensional observation feature information into a new fully connected neural network branch and outputs the attack strategy distribution in the corresponding attack sub-action space. By sampling the sub-attack strategy distributions output by the actor network, the sub-attack actions of a single attack can be obtained, and these can be assembled to obtain a complete network security attack. In this scheme, the attack initiating node of a network security attack is randomly selected from network nodes that have already obtained local host privileges.

[0095] In automated penetration testing scenarios, numerous invalid attack actions exist. For example, a network security attack cannot be launched if no discovered network nodes exist in the target test network; a remote attack cannot be launched if no network nodes with local host privileges exist in the target test network, and so on. To avoid sampling invalid attack actions from the actor network, this solution proposes an action mask mechanism to handle invalid attack actions in automated penetration testing scenarios. Based on the agent's original observation information, the action mask of the agent's current state is calculated. The action mask is a set of binary vectors applied to the main attack action, with the same dimensions as the main attack action. In the current state, invalid attack sub-actions in the agent's main attack action will have their corresponding dimensions set to zero in the action mask; valid attack sub-actions will have their corresponding dimensions set to one in the action mask. For attack sub-action dimensions with zero values ​​in the action mask, the selection probability of the corresponding attack sub-action dimension in the main attack action policy distribution output by the actor network is set to zero, thereby avoiding sampling invalid attack sub-actions from the actor network.

[0096] The pre-defined automated penetration testing agent can output attack sub-actions in various dimensions, and process these sub-actions using sub-action mask information to obtain a pre-defined probability distribution for each attack sub-action. This pre-defined probability distribution is an effective probability distribution.

[0097] In practice, random decision-making can be used to filter attack sub-actions in each dimension through a preset probability distribution, thereby selecting the target attack sub-actions.

[0098] It should be noted that there is a dependency between the two attack sub-actions in the main attack action. The effective attack sub-action dimension in the attack type sub-action space is uniquely affected by the agent's observation information, while the attack type sub-action affects the effective attack sub-action dimension in the target node sub-action space. Therefore, during execution, the action mask is first applied to the attack type sub-action space, outputting the attack type sub-action decision. Then, based on the output attack type sub-action, the action mask is updated, and the updated action mask is applied again to the target node sub-action space, outputting the target node sub-action decision.

[0099] In practice, the target attack sub-actions are the various sub-actions of this attack. The target attack actions can be obtained by assembling the attack weapon arsenal and the target attack sub-actions.

[0100] For a specific primary attack action decision, only one attack sub-action space is valid in the attack parameter space (target local vulnerability for local attack, target remote vulnerability for remote attack, target port for lateral movement). Therefore, when assembling a complete cybersecurity attack, only the primary attack action and the valid attack sub-actions in the attack parameters need to be assembled; other invalid attack sub-actions in the attack parameters will be discarded. The selection of target local vulnerabilities, target remote vulnerabilities, and target ports depends on the arsenal of the automated penetration testing agent and can be collected in advance through various methods.

[0101] This embodiment obtains real network environment information and corresponding sub-action mask information based on the initial information of the real network environment; inputs the real network environment information and the sub-action mask information into a preset automated penetration testing agent to obtain the target attack action; based on the observed target network environment information, outputs different aspects of each attack sub-action at the current decision moment, and combines them into a complete attack action, thereby realizing fine control of attack behavior during the penetration testing process.

[0102] refer to Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the automated penetration testing method of the present invention.

[0103] Based on the first embodiment described above, step S40 of the automated penetration testing method in this embodiment includes:

[0104] Step S401: Obtain the updated network environment information and the corresponding update sub-action mask based on the feedback information.

[0105] It should be noted that the target network environment can provide feedback on updated network environment information and corresponding update sub-action masks. Therefore, new network environment information and new sub-action mask information can be obtained through the feedback information.

[0106] Step S402: Use the updated network environment information and the updated sub-action mask as the initial information for updating the real network environment, and return to the step of inputting the initial information of the real network environment into the preset automated penetration testing agent to obtain the target attack action, until the attack target or number of attacks in the target network environment meets the preset number of attacks.

[0107] In practice, the updated network environment information and the updated sub-action mask information can be used as the initial information for updating the real network environment. The initial information of the real network environment is then re-executed and input into the preset automated penetration testing agent. The preset automated penetration testing agent then decides on the complete attack action and uses the updated initial information of the real network environment as the input for the next attack decision until the attack target of the target network environment is achieved or the set number of attacks is reached. The preset number of attacks is the maximum number of attacks.

[0108] This embodiment obtains updated network environment information and corresponding update sub-action masks based on the feedback information; uses the updated network environment information and the update sub-action masks as initial information for updating the real network environment, and returns to the step of inputting the initial information of the real network environment into the preset automated penetration testing agent to obtain the target attack action, until the attack target or number of attacks in the target network environment meets the preset number of attacks, thereby continuously updating the attack and improving the testing accuracy and efficiency of the generated automated penetration testing model.

[0109] refer to Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the automated penetration testing method of the present invention.

[0110] Based on the first embodiment described above, the automated penetration testing method of this embodiment further includes, before step S20:

[0111] Step S11: Collect penetration test sample log data.

[0112] It should be noted that before conducting automated penetration testing, sample log data of the penetration testing can be collected first, so as to train the initial automated penetration testing agent and build a preset automated penetration testing agent.

[0113] Optionally, the steps of collecting penetration test sample log data specifically include: collecting real attack data and a first attack result; obtaining the attack strategy and a second attack result of the online automated penetration test agent; constructing a network environment and performing an attack in the network environment using a preset attack strategy, recording sample attack actions and a third attack result; and obtaining test sample log data through the real attack data, the first attack result, the attack strategy, the second attack result, the sample attack actions, and the third attack result.

[0114] It should be noted that, to accelerate the learning process of the pre-set automated penetration testing agent, a large amount of penetration testing sample log data needs to be collected first. The collection methods are as follows: First, collect real attack data and initial attack results. Real attack data refers to the hacker's attack data and the corresponding initial attack results. Second, record the attack strategy selection and results of the online automated penetration testing agent. Third, construct a simulated or real network environment and perform attacks using the pre-set attack strategy within that environment to obtain sample attack actions and final attack results.

[0115] The default attack strategy is a heuristic attack strategy. The third method is as follows: reset the network environment and obtain the initial state information of the network environment to prepare for subsequent evaluation of the attack results. Then, use the heuristic attack strategy to perform attack actions, including random selection and simple heuristic algorithms, and observe the network state, record the state and feedback, thereby obtaining sample attack actions and the third attack result.

[0116] Heuristic penetration testing strategies, such as random penetration testing strategies and online reinforcement learning-based penetration testing strategies, can be used to continuously attempt penetration tests in the network environment. Offline penetration test sample log data can be collected through interaction with the target network environment, which can be used to train automated penetration testing agents offline.

[0117] In practice, the steps of the third method can be repeated continuously to collect a large number of sample attack actions and third attack results. Test sample log data can be obtained by combining real attack data, first attack results, attack strategies, second attack results, sample attack actions, and third attack results.

[0118] Step S12: Train the initial automated penetration testing agent using the penetration test sample log data to construct a preset automated penetration testing agent.

[0119] It should be noted that the initial automated penetration testing agent can be trained using penetration test sample log data, thereby constructing a preset automated penetration testing agent.

[0120] Specifically, the step of training an initial automated penetration testing agent using the penetration testing sample log data to construct a preset automated penetration testing agent includes: training the initial automated penetration testing agent using a preset reinforcement learning strategy based on the penetration testing sample log data to obtain a reference automated penetration testing agent; performing penetration testing performance and efficiency tests on the reference automated penetration testing agent in a real network environment to obtain test results; and adjusting the parameters of the reference automated penetration testing agent according to the test results to construct the preset automated penetration testing agent.

[0121] Understandably, a simulated or real network environment can be built first to provide training data for automated penetration testing agents.

[0122] In specific implementation, the preset reinforcement learning strategy can be an offline reinforcement learning algorithm. The optional offline reinforcement learning algorithms include, but are not limited to: conservative reinforcement learning algorithm, twin delay depth-determined policy gradient + behavior cloning algorithm, and behavior proximal policy optimization algorithm.

[0123] An initial automated penetration testing agent can be trained using a pre-defined reinforcement learning strategy, thereby enabling it to learn effective penetration testing strategies and obtain a reference automated penetration testing agent.

[0124] In practical implementation, the reference automated penetration testing agent can be verified in a real network environment. Specifically, the penetration testing performance and efficiency of the reference automated penetration testing agent can be tested to obtain test results. Based on the test results, the reference automated penetration testing agent can be adjusted and continuously optimized to build a preset automated penetration testing agent.

[0125] like Figure 7 As shown, Figure 7 This is a schematic diagram of the overall process of automated penetration testing. It involves collecting offline training data samples, specifically through network environment systems and heuristic attack strategies. The collected penetration test data samples are then input into the initial automated penetration test agent for offline reinforcement sequences, thereby constructing a preset automated penetration test agent and deploying it to the target test network for automated penetration testing.

[0126] This embodiment collects penetration test sample log data; trains an initial automated penetration test agent using the penetration test sample log data to construct a preset automated penetration test agent; uses an offline reinforcement learning algorithm to guide the learning process of the automated penetration test model, and creates an automated penetration test learning model based on offline reinforcement learning for autonomously learning penetration test behavior patterns, thereby improving the efficiency of subsequent automated penetration tests.

[0127] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the automated penetration testing device of the present invention.

[0128] like Figure 8 As shown, the automated penetration testing device proposed in this embodiment of the invention includes:

[0129] Module 10 is used to acquire initial information about the real network environment.

[0130] The input module 20 is used to input the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action.

[0131] The loading module 30 is used to load the target attack action into the target network environment and receive feedback information from the target network environment.

[0132] The testing module 40 is used to perform automated penetration testing based on the feedback information.

[0133] This embodiment obtains initial information about a real network environment; inputs this initial information into a preset automated penetration testing agent to obtain target attack actions; loads the target attack actions into the target network environment and receives feedback information from the target network environment; and performs automated penetration testing based on the feedback information. This solution for automated penetration testing does not overly rely on human resources, enabling automated execution and fine-grained control of the entire sequential penetration testing process, improving the accuracy and adaptability of the test, and thus more effectively identifying potential security vulnerabilities and threats.

[0134] In one embodiment, the input module 20 is further configured to obtain real network environment information and corresponding sub-action mask information based on the initial information of the real network environment; input the real network environment information and the sub-action mask information into a preset automated penetration testing agent to obtain the target attack action.

[0135] In one embodiment, the input module 20 is further configured to input the real network environment information and the sub-action mask information into a preset automated penetration testing agent, process the real network environment information through the preset automated penetration testing agent, and output attack sub-actions of various dimensions; process the attack sub-actions of various dimensions through the sub-action mask information to obtain a preset probability distribution of attack sub-actions of various dimensions; use random decision-making based on the preset probability distribution to filter the attack sub-actions of various dimensions to obtain target attack sub-actions; and assemble the target attack sub-actions to obtain the target attack action.

[0136] In one embodiment, the testing module 40 is further configured to obtain updated network environment information and corresponding update sub-action mask based on the feedback information; use the updated network environment information and the update sub-action mask as initial information for updating the real network environment, and return to the step of inputting the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action, until the attack target or number of attacks in the target network environment meets the preset number of attacks.

[0137] In one embodiment, the input module 20 is further configured to collect penetration test sample log data; and to train an initial automated penetration test agent using the penetration test sample log data to construct a preset automated penetration test agent.

[0138] In one embodiment, the input module 20 is further configured to collect real attack data and a first attack result; obtain the attack strategy and a second attack result of the online automated penetration testing agent; construct a network environment and perform an attack using a preset attack strategy in the network environment, and record sample attack actions and a third attack result; and obtain test sample log data through the real attack data, the first attack result, the attack strategy, the second attack result, the sample attack action, and the third attack result.

[0139] In one embodiment, the input module 20 is further configured to train an initial automated penetration testing agent using a preset reinforcement learning strategy based on the penetration testing sample log data to obtain a reference automated penetration testing agent; perform penetration testing performance and efficiency tests on the reference automated penetration testing agent in a real network environment to obtain test results; and adjust the parameters of the reference automated penetration testing agent according to the test results to construct a preset automated penetration testing agent.

[0140] Furthermore, embodiments of the present invention also propose a storage medium storing an automated penetration testing program, which, when executed by a processor, implements the steps of the automated penetration testing method described above.

[0141] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0142] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0143] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0144] In addition, for technical details not described in detail in this embodiment, please refer to the automated penetration testing method provided in any embodiment of the present invention, which will not be repeated here.

[0145] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0146] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0148] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An automated penetration testing method, characterized in that, The automated penetration testing method includes: Obtain initial information about the real network environment; The initial information of the real network environment is input into a preset automated penetration testing agent to obtain the target attack action; The target attack action is loaded into the target network environment, and feedback information from the target network environment is received. Automated penetration testing is performed based on the feedback information; Before inputting the initial information of the real network environment into the preset automated penetration testing agent to obtain the target attack action, the process further includes: Collect penetration test sample log data, including: collect real attack data and the results of the first attack; The system acquires the attack strategy and second attack result of the online automated penetration testing agent; constructs a network environment and performs an attack in the network environment using a preset attack strategy, recording sample attack actions and a third attack result; and obtains test sample log data through the real attack data, the first attack result, the attack strategy, the second attack result, the sample attack actions, and the third attack result. Training an initial automated penetration testing agent using the penetration test sample log data to construct a preset automated penetration testing agent includes: training the initial automated penetration testing agent using a preset reinforcement learning strategy based on the penetration test sample log data to obtain a reference automated penetration testing agent; performing penetration testing performance and efficiency tests on the reference automated penetration testing agent in a real network environment to obtain test results; and adjusting the parameters of the reference automated penetration testing agent based on the test results to construct the preset automated penetration testing agent.

2. The automated penetration testing method as described in claim 1, characterized in that, The step of inputting the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action includes: Based on the initial information of the real network environment, obtain the real network environment information and the corresponding sub-action mask information; The real network environment information and the sub-action mask information are input into a preset automated penetration testing agent to obtain the target attack action.

3. The automated penetration testing method as described in claim 2, characterized in that, The step of inputting the real network environment information and the sub-action mask information into a preset automated penetration testing agent to obtain the target attack action includes: The real network environment information and the sub-action mask information are input into a preset automated penetration testing agent. The preset automated penetration testing agent processes the real network environment information and outputs attack sub-actions of various dimensions. The attack sub-actions of each dimension are processed by the sub-action mask information to obtain the preset probability distribution of each attack sub-action. The target attack sub-action is obtained by using random decision-making through the preset probability distribution to filter the attack sub-actions of each dimension. The target attack sub-actions are assembled to obtain the target attack action.

4. The automated penetration testing method as described in claim 1, characterized in that, The automated penetration testing based on the feedback information includes: Based on the feedback information, the updated network environment information and the corresponding update sub-action mask are obtained; The updated network environment information and the updated sub-action mask are used as the initial information for updating the real network environment. The process of inputting the initial information of the real network environment into the preset automated penetration testing agent to obtain the target attack action continues until the attack target or number of attacks in the target network environment meets the preset number of attacks.

5. An automated penetration testing device, characterized in that, The automated penetration testing device performs the automated penetration testing method according to any one of claims 1 to 4, and the automated penetration testing device comprises: The acquisition module is used to acquire initial information about the real network environment; The input module is used to input the initial information of the real network environment into a preset automated penetration testing agent to obtain the target attack action; The loading module is used to load the target attack action into the target network environment and receive feedback information from the target network environment. The testing module is used to perform automated penetration testing based on the feedback information.

6. An automated penetration testing device, characterized in that, The automated penetration testing device includes: a memory, a processor, and an automated penetration testing program stored in the memory and executable on the processor, the automated penetration testing program being configured to implement the automated penetration testing method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium stores an automated penetration testing program, which, when executed by a processor, implements the automated penetration testing method as described in any one of claims 1 to 4.

Images