Automated penetration testing method, apparatus, device, and storage medium

Abstract

The application relates to the technical field of network security, and discloses an automatic penetration testing method, device, equipment and storage medium, the method comprising the following steps: collecting executable attack codes, and generating an attack weapon library based on the executable attack codes; inputting real network information corresponding to a real network environment into an automatic penetration testing decision model to obtain real attack sub-actions, wherein the automatic penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology; and performing automatic penetration testing on the real network environment based on the attack weapon library and the real attack sub-actions. Since the automatic penetration testing decision model is used to obtain real attack sub-actions suitable for the real network environment, and the real attack sub-actions and the executable attack codes in the attack weapon library are combined to perform penetration testing on the real network environment, the network environment can be quickly and accurately subjected to automatic penetration testing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to an automatic penetration testing method, device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of technologies such as the Internet, big data and the Internet of Things, more and more devices are connected to the Internet. However, a large number of devices have security vulnerabilities and lack effective defense measures. Therefore, these devices have become the target of malicious attackers. In order to improve network security from the perspective of the defender, security personnel have proposed penetration testing. Penetration testing is a technology for discovering and exploiting vulnerabilities in computer systems, which can simulate the attack behavior of attackers and automatically test and assess the security of target systems to handle or avoid possible security vulnerabilities and risks.

[0003] The existing penetration testing usually generates a penetration testing strategy through manual penetration testing by professional testers, so as to perform penetration testing on the current network environment. However, this method may cause false alarms and missed vulnerabilities in actual testing scenarios, and even experienced testers cannot guarantee the reliability and accuracy of penetration testing. In addition, since manual penetration testing cannot autonomously generate new penetration testing strategies, it needs to be configured, executed, monitored and maintained manually to generate a penetration testing strategy, which requires a large amount of time and human resources, and the efficiency of penetration testing is low. Based on the above discussion, there is an urgent need in the industry for a method that can quickly and accurately perform automatic penetration testing on a network environment.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide an automatic penetration testing method, device, equipment and storage medium, which aims to solve the technical problem that the prior art cannot quickly and accurately perform automatic penetration testing on a network environment.

[0006] To achieve the above purpose, the present application provides an automatic penetration testing method, which comprises the following steps:

[0007] Collecting executable attack codes and generating attack weapon libraries based on the executable attack codes;

[0008] Inputting real network information corresponding to a real network environment into an automatic penetration testing decision model to obtain real attack sub-actions, wherein the automatic penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology;

[0009] based on the attack weapon library and the real attack sub-action, performing an automatic penetration test on the real network environment.

[0010] Optionally, before the step of inputting the real network information corresponding to the real network environment into the automatic penetration test decision model to obtain the real attack sub-action, the method further comprises:

[0011] based on the initial penetration test model and the attack weapon library, performing attack training on the target environment network, and obtaining a feedback reward value returned by the target environment network after being attacked;

[0012] If the feedback reward value reaches a preset threshold, it is observed whether the performance index of the initial penetration test model is stably converged, and if so, the initial penetration test model is determined as the automatic penetration test decision model.

[0013] Optionally, after the step of based on the initial penetration test model and the attack weapon library, performing attack training on the target environment network, and obtaining a feedback reward value returned by the target environment network after being attacked, the method further comprises:

[0014] If the performance index of the initial penetration test model is not stably converged, the step of based on the initial penetration test model and the attack weapon library, performing attack training on the target environment network is repeatedly executed until the performance index of the initial penetration test model is stably converged.

[0015] Optionally, the step of based on the initial penetration test model and the attack weapon library, performing attack training on the target environment network, comprises:

[0016] resetting the target environment network and obtaining target initial information corresponding to the target environment network;

[0017] inputting the target initial information into the initial penetration test model to obtain a target attack sub-action;

[0018] based on the attack weapon library and the target attack sub-action, performing attack on the target environment network.

[0019] Optionally, the step of inputting the real network information corresponding to the real network environment into the automatic penetration test decision model to obtain the real attack sub-action, comprises:

[0020] inputting the real network information corresponding to the real network environment into the automatic penetration test decision model to obtain a sub-action mask of each dimension attack sub-action and a probability distribution of each dimension attack sub-action;

[0021] determine an effective probability distribution of the attack sub-actions based on the sub-action mask and the probability distribution of the attack sub-actions, and select a real attack sub-action based on the effective probability distribution.

[0022] Optionally, the step of performing the automatic penetration test on the real network environment based on the attack weapon library and the real attack sub-action comprises:

[0023] Assembling a complete attack action based on the attack weapon library and the real attack sub-action, wherein the complete attack action comprises an attack target, an attack form, and specific attack parameters.

[0024] Performing the automatic penetration test on the real network environment according to the complete attack action, to obtain updated network environment information corresponding to the updated real network environment and an updated feedback reward value.

[0025] Determining whether a test result of the automatic penetration test meets a preset condition according to the updated network environment information and the updated feedback reward value.

[0026] If yes, ending the current round of the automatic penetration test.

[0027] Optionally, after the step of determining whether the test result of the automatic penetration test meets the preset condition according to the updated network environment information and the updated feedback reward value, the method further comprises:

[0028] If no, performing the following steps until the test result meets the preset condition:

[0029] Inputting real network information corresponding to the real network environment into an automatic penetration test decision model to obtain a real attack sub-action.

[0030] Assembling a complete attack action based on the attack weapon library and the real attack sub-action, wherein the complete attack action comprises an attack target, an attack form, and specific attack parameters.

[0031] Performing the automatic penetration test on the real network environment according to the complete attack action, to obtain updated network environment information corresponding to the updated real network environment and an updated feedback reward value.

[0032] Determining whether a test result of the automatic penetration test meets a preset condition according to the updated network environment information and the updated feedback reward value.

[0033] In addition, to achieve the above object, the application further provides an automatic penetration test device, which comprises:

[0034] a data collection module configured to collect executable attack codes and generate an attack weapon library based on the executable attack codes;

[0035] an action generation module configured to input real network information corresponding to a real network environment into an automated penetration testing decision model to obtain real attack sub-actions, wherein the automated penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology;

[0036] a penetration testing module configured to perform automated penetration testing on the real network environment based on the attack weapon library and the real attack sub-actions.

[0037] In addition, to achieve the above object, the present application further provides an automated penetration testing device, which comprises a memory, a processor, and an automated penetration testing program stored in the memory and executable on the processor, wherein the automated penetration testing program is configured to implement the steps of the above-mentioned automated penetration testing method.

[0038] In addition, to achieve the above object, the present application further provides a storage medium having an automated penetration testing program stored thereon, wherein the automated penetration testing program, when executed by a processor, implements the steps of the above-mentioned automated penetration testing method.

[0039] The present application collects executable attack codes, generates an attack weapon library based on the executable attack codes, inputs real network information corresponding to a real network environment into an automated penetration testing decision model to obtain real attack sub-actions, wherein the automated penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology, and performs automated penetration testing on the real network environment based on the attack weapon library and the real attack sub-actions. Compared with the prior art which generates a penetration testing strategy through manual penetration testing, the present application pre-trains an initial penetration testing model based on deep reinforcement learning technology to obtain an automated penetration testing decision model, inputs real network information corresponding to a real network environment into the automated penetration testing decision model to obtain real attack sub-actions, and then performs penetration testing on the real network environment in combination with the real attack sub-actions and executable attack codes in the attack weapon library, so that a new penetration testing strategy can be autonomously generated in a scenario without relying on professional testers, and the network environment can be quickly and accurately tested by automated penetration testing. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 FIG. 1 is a structural schematic diagram of an automated penetration testing device for a hardware running environment according to an embodiment of the present application;

[0041] Figure 2The flowchart of the first embodiment of the automatic penetration test method of the present application is shown in the figure.

[0042] Figure 3 The flowchart of the second embodiment of the automatic penetration test method of the present application is shown in the figure.

[0043] Figure 4 The flowchart of the third embodiment of the automatic penetration test method of the present application is shown in the figure.

[0044] Figure 5 The structural block diagram of the first embodiment of the automatic penetration test device of the present application is shown in the figure.

[0045] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are intended to explain the present application and are not intended to limit the present application.

[0047] Reference Figure 1 , Figure 1 The structural diagram of the automatic penetration test device related to the hardware running environment of the embodiment of the present application is shown in the figure.

[0048] As Figure 1 shown, the automatic penetration test device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 can be a high-speed random access memory (RAM), and can also be a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0049] Those skilled in the art can understand that Figure 1 the structure shown in the figure does not constitute a limitation on the automatic penetration test device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0050] As shown in Figure 1 , the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and an automated penetration testing program.

[0051] In Figure 1 the automated penetration testing device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the automated penetration testing device of the present application can be arranged in the automated penetration testing device, and the automated penetration testing device calls the automated penetration testing program stored in the memory 1005 through the processor 1001, and executes the automated penetration testing method provided in the embodiments of the present application.

[0052] The embodiments of the present application provide an automated penetration testing method, with reference to Figure 2 , Figure 2 for the flowchart of the first embodiment of the automated penetration testing method of the present application.

[0053] In this embodiment, the automated penetration testing method includes the following steps:

[0054] Step S10: Collect executable attack codes and generate an attack weapon library based on the executable attack codes.

[0055] It should be noted that the execution subject of the method of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a mobile phone, a tablet computer, a personal computer, etc., and can also be other electronic devices capable of achieving the same or similar functions, which are not limited in the present embodiment. Hereinafter, the automated penetration testing device (hereinafter referred to as testing device) will be taken as an example to describe the various embodiments of the automated penetration testing method of the present application.

[0056] It can be understood that the executable attack code is a series of code scripts capable of attacking the network environment. The executable attack code can be divided into various types, such as worm viruses, Trojans, etc.

[0057] In specific implementation, the executable attack codes can be collected in various ways to generate an attack weapon library. For example, the executable attack codes provided by the contestants in the competition can be collected by holding an information security competition, and open source executable attack codes can be searched and collected from the Internet, etc., and the present embodiment does not limit the collection of executable attack codes.

[0058] Step S20: inputting real network information corresponding to a real network environment into an automatic penetration testing decision model to obtain a real attack sub-action, wherein the automatic penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology.

[0059] It should be noted that the real network environment mentioned above is a network environment that needs to perform automatic penetration testing, and the real network information mentioned above can include initial network information (such as IP address, network topology, operating system, sub-action mask, etc.) of the real network environment.

[0060] It should be understood that the automatic penetration testing decision model mentioned above can be a model capable of executing different penetration testing strategies based on different network environment information.

[0061] In a specific implementation, the initial penetration testing model based on the Multi-Head architecture of deep reinforcement learning can use multiple different output heads to respectively decide each-dimensional attack sub-action, so that the initial penetration testing model has the ability to automatically sequentially decide attack behavior and finely control attack behavior, and learns effective penetration testing strategies through exploration and utilization in the target environment network, and finally outputs a trained initial penetration testing learning model, which is the automatic penetration testing decision model mentioned above. The training scene can be based on a simulated or real target environment network.

[0062] Step S30: performing automatic penetration testing on the real network environment based on the attack weapon library and the real attack sub-action.

[0063] In a specific implementation, the automatic penetration testing mentioned above can simulate the attack behavior of an attacker, and perform security testing and risk assessment on the target environment network in an automatic manner to handle or avoid possible security vulnerabilities and risks, so as to repair vulnerabilities before network attacks, thereby improving the security of the network.

[0064] Further, in the present embodiment, in order to determine a more effective real attack sub-action according to the real network environment, thereby improving the testing effect of penetration testing, the step S20 can include:

[0065] Step S201: inputting real network information corresponding to a real network environment into an automatic penetration testing decision model to obtain a sub-action mask of each-dimensional attack sub-action and a probability distribution of each-dimensional attack sub-action.

[0066] It should be noted that the above sub-action mask can be a string of binary codes set to 1 or 0, used to indicate which of the attack sub-actions in each dimension are valid and executable, and which are invalid and unexecutable.

[0067] Step S202: determining an effective probability distribution of the attack sub-actions in each dimension based on the sub-action mask and the probability distribution of the attack sub-actions in each dimension, and selecting a real attack sub-action based on the effective probability distribution.

[0068] It should be understood that due to the variety of network attack actions, there can be several invalid attack network attack actions (such as launching an attack in the form of an invalid attack on the attack target, or assembling an attack action using mismatched attack forms and attack parameters, etc.) in the same network environment, and of course there can be several network attack actions that can effectively attack the network. If the network environment is attacked one by one in sequence, the attack efficiency will be low, which will affect the efficiency of penetration testing. If the network environment is attacked in batches, data congestion may occur, and even the network environment may directly malfunction. Therefore, in this embodiment, in order to shield the interference and noise caused by invalid attack sub-actions in the attack decision process, the sub-action mask of the effective attack sub-action can be calculated based on the observed network environment information and the selected attack sub-action, to shield the probability of invalid attack sub-actions output by the initial penetration testing model. The calculated sub-action mask will act on the probability distribution of each sub-action space output by the automated penetration testing decision model, thereby obtaining the effective probability distribution of the attack sub-actions in each dimension in the attack decision, and then selecting a real attack sub-action based on the corresponding effective probability distribution, thereby ensuring the randomness and diversity of the attack action.

[0069] The embodiment collects executable attack codes, generates an attack weapon library based on the executable attack codes, inputs real network information corresponding to a real network environment into an automatic penetration testing decision model to obtain a sub-action mask of each dimension attack sub-action and a probability distribution of each dimension attack sub-action, determines an effective probability distribution of each dimension attack sub-action based on the sub-action mask and the probability distribution of each dimension attack sub-action, and decides to select a real attack sub-action based on the effective probability distribution. The automatic penetration testing method of the embodiment is compared with the penetration testing strategy generated by the artificial penetration testing method of the prior art. Since the initial penetration testing model is trained based on the deep reinforcement learning technology in advance to obtain the automatic penetration testing decision model, and the real attack sub-action is obtained by inputting the real network information corresponding to the real network environment into the automatic penetration testing decision model, and the real network environment is penetrated by combining the real attack sub-action and the executable attack codes in the attack weapon library, the new penetration testing strategy can be autonomously generated in the scene without relying on professional testers, and the network environment can be quickly and accurately penetrated by the automatic penetration testing method.

[0070] Reference Figure 3 , Figure 3 FIG. 2 is a flowchart of a second embodiment of the automatic penetration testing method of the present application.

[0071] Based on the first embodiment, in the second embodiment, in order to obtain an automatic penetration testing decision model with convergent and stable performance, thereby improving the reliability of the output result of the automatic penetration testing decision model, the step S20 can further include the following steps before the step S20:

[0072] Step S11: performing attack training on a target environment network based on the initial penetration testing model and the attack weapon library, and obtaining a feedback reward value returned by the target environment network after being attacked.

[0073] It should be noted that the feedback reward value can be simply understood as a reference value for judging whether the attack is effective after the target environment network is attacked. Specifically, the feedback reward value can be divided into positive feedback reward and negative feedback reward: if an attack achieves effective vulnerability exploitation, a positive feedback reward can be output as an encouragement of the effective attack action; if an attack fails, a negative feedback reward can be output to suppress the selection of the failed attack action. Based on this, the feedback reward value can be used as a reward signal to guide the learning direction of the initial penetration testing model in the attack training process.

[0074] It should be understood that the above attack training process can be implemented based on a deep reinforcement learning algorithm. Among them, the optional deep reinforcement learning algorithm can be a deep Q-network (DQN) algorithm and its derivative algorithms, including DDQN, Dueling DQN, D3QN, etc., or an asynchronous advantage actor-critic (A3C) algorithm, a proximal policy optimization (PPO) algorithm, etc., which is not limited by the embodiments.

[0075] In a specific implementation, the specific value of the positive and negative feedback reward can be designed in more detail based on the impact of the attack. For example, finding a vulnerability on a host node that can enhance the local user's authority is more valuable and meaningful than finding a host node in the target network system, so it should get a larger positive feedback reward value, and the value of the negative feedback reward can also be shaped in a similar way. When the preset attack target or the upper limit of the preset attack step is reached during the attack process, the round sequential decision process ends, and a positive or negative termination reward feedback is obtained based on the termination state.

[0076] Step S12: If the feedback reward value reaches a preset threshold, observe whether the performance indicator of the initial penetration test model is stably convergent, and if so, determine the initial penetration test model as an automated penetration test decision model.

[0077] It should be noted that the performance indicator of the initial penetration test model can be the attack step used to complete the attack target, or the cumulative reward value or other indicators to complete the attack target, which is not limited by the embodiments.

[0078] Further, in the present embodiment, in order to make the performance indicator of the initial penetration test model continue to attack the target environment network in the case of unstable convergence, after the step S12, it can also include:

[0079] Step S13: If the performance indicator of the initial penetration test model is not stably convergent, repeat the step of attacking the target environment network based on the initial penetration test model and the attack weapon library for attack training until the performance indicator of the initial penetration test model is stably convergent.

[0080] In a specific implementation, whether the performance indicator of the initial penetration testing model is stably convergent can be determined by judging whether the performance indicator of the initial penetration testing model reaches the preset performance indicator. For example, after the initial penetration testing model performs a complete attack sequence, if the performance indicator (such as the attack step length used to complete the attack target, the cumulative reward value for completing the attack target, etc.) of the current initial penetration testing model reaches the attack step length, the cumulative reward value, etc. required in the preset performance indicator, it can be determined that the performance indicator of the initial penetration testing model is stably convergent. If the performance indicator of the initial penetration testing model does not reach the preset performance indicator, it can be determined that the performance indicator of the initial penetration testing model is not stably convergent.

[0081] Further, in the present embodiment, in order to improve the training effect of attack training and thus more rapidly build an automated penetration testing decision model, the step S11 can include:

[0082] Step S111: resetting the target environment network and obtaining target initial information corresponding to the target environment network.

[0083] It should be noted that the target environment network is a network environment that needs to perform attack training, and the target initial information can include initial network information (such as IP address, network topology, operating system, sub-action mask, etc.) of the target network environment.

[0084] It should be understood that, in order to avoid the influence of historical attacks on current attacks or subsequent attacks and thus generate more effective attack actions, the target environment network can be reset before each round of attack training starts to ensure the effectiveness of attack training.

[0085] Step S112: inputting the target initial information into the initial penetration testing model to obtain a target attack sub-action.

[0086] In a specific implementation, the target environment network can be attacked by the initial penetration testing model, so as to learn the behavior pattern of real network attacks, quickly perform new attack decisions based on observed network environment information, and realize complete automation and fine-grained attack control of the entire penetration testing process.

[0087] Step S113: attacking the target environment network based on the attack weapon library and the target attack sub-action.

[0088] In a specific implementation, the behavior pattern of an attacker can be imitated by means of the attack weapon library, the target attack sub-action, and the initial penetration testing model that continuously implements deep reinforcement learning to attack, so as to automatically select the attack target, the attack form, and the specific attack parameters, etc. in each attack in the sequential attack process.

[0089] The embodiment trains the target environment network based on the initial penetration test model and the attack weapon library, and obtains a feedback reward value returned by the target environment network after being attacked; if the feedback reward value reaches a preset threshold, it is observed whether the performance index of the initial penetration test model is stably converged, if yes, the initial penetration test model is determined as the automatic penetration test decision model; if the performance index of the initial penetration test model is not stably converged, the step of training the target environment network based on the initial penetration test model and the attack weapon library is repeatedly executed until the performance index of the initial penetration test model is stably converged. Compared with the prior art of generating a penetration test strategy by manual penetration test, the above method of the embodiment introduces a feedback reward value as a reward signal to guide the learning direction of the initial penetration test model in the attack training process, so that an automatic penetration test decision model with stable performance is obtained, and the credibility of the output result of the automatic penetration test decision model is improved.

[0090] Reference Figure 4 , Figure 4 The flowchart of the third embodiment of the automatic penetration test method of the application is shown.

[0091] Based on the above embodiments, in the embodiment, in order to improve the reliability of the automatic penetration test, so as to accurately find out the effective attack action, the step S30 can include:

[0092] Step S301: Assemble a complete attack action based on the attack weapon library and the real attack sub-action, the complete attack action including an attack target, an attack form and specific attack parameters.

[0093] In a specific implementation, the corresponding output head of the deep neural network model with a Multi-Head structure can be used for decision-making respectively, and the attack weapon corresponding to the attack weapon library is selected based on the decision result, and then the attack weapon and the real attack sub-action are assembled to obtain the complete attack action.

[0094] Step S302: Perform automatic penetration test on the real network environment according to the complete attack action, and obtain updated network environment information and an updated feedback reward value corresponding to the updated real network environment.

[0095] It should be noted that the updated feedback reward value is also divided into an updated positive feedback reward value and an updated negative feedback reward value, and the related principles are the same as those of the positive feedback reward value and the negative feedback reward value in the second embodiment, which will not be repeated here.

[0096] Step S303: determining whether the test result of the automated penetration testing meets a preset condition according to the updated network environment information and the updated feedback reward value.

[0097] It should be noted that the preset condition can be that the current attack is an effective attack or the current attack reaches a set maximum attack step length.

[0098] In a specific implementation, whether the current attack is an effective attack or reaches a set maximum attack step length can be determined by the updated network environment information and the updated feedback reward value. Specifically, if the current attack is an effective attack or reaches a set maximum attack step length, it can be determined that the test result of the automated penetration testing meets the preset condition; if the current attack is not an effective attack and does not reach a set maximum attack step length, it can be determined that the test result of the automated penetration testing does not meet the preset condition.

[0099] Step S304: if yes, ending the current round of automated penetration testing.

[0100] Further, in the embodiment, in order to continue to attack the real network environment when no effective attack action is found or the maximum attack step length is not reached, thereby ensuring the test effect of the penetration testing of the embodiment, after the step S303, the following step can also be included:

[0101] Step S305: if no, performing the following steps until the test result meets the preset condition: inputting real network information corresponding to the real network environment into the automated penetration testing decision model to obtain a real attack sub-action; assembling a complete attack action based on the attack weapon library and the real attack sub-action, the complete attack action including an attack target, an attack form, and specific attack parameters; performing automated penetration testing on the real network environment according to the complete attack action to obtain updated network environment information corresponding to an updated real network environment and an updated feedback reward value; and determining whether the test result of the automated penetration testing meets the preset condition according to the updated network environment information and the updated feedback reward value.

[0102] The embodiment assembles a complete attack action based on the attack weapon library and the real attack sub-action, and the complete attack action includes an attack target, an attack form and specific attack parameters; performs automatic penetration testing on the real network environment according to the complete attack action, obtains updated network environment information corresponding to the updated real network environment and an updated feedback reward value; determines whether the test result of the automatic penetration testing meets a preset condition according to the updated network environment information and the updated feedback reward value; if yes, the round of automatic penetration testing is ended; if no, the following steps are performed until the test result meets the preset condition: inputs the real network information corresponding to the real network environment into the automatic penetration testing decision model to obtain a real attack sub-action; assembles a complete attack action based on the attack weapon library and the real attack sub-action, and the complete attack action includes an attack target, an attack form and specific attack parameters; performs automatic penetration testing on the real network environment according to the complete attack action, obtains updated network environment information corresponding to the updated real network environment and an updated feedback reward value; and determines whether the test result of the automatic penetration testing meets a preset condition according to the updated network environment information and the updated feedback reward value. Compared with the prior art that generates a penetration testing strategy by means of manual penetration testing, the above-mentioned method of the embodiment determines the test result of the automatic penetration testing according to the updated network environment information and the updated feedback reward value, and can automatically continue to attack the real network environment when the test result does not meet the preset condition, thereby ensuring the test effect of the penetration testing of the embodiment.

[0103] In addition, the embodiment of the present application further provides a storage medium, wherein the storage medium stores an automatic penetration testing program, and the automatic penetration testing program is executed by a processor to realize the steps of the automatic penetration testing method as described above.

[0104] Reference Figure 5 , Figure 5 is a structural block diagram of the first embodiment of the automatic penetration testing device of the present application.

[0105] As Figure 5 shown, the automatic penetration testing device provided by the embodiment of the present application comprises:

[0106] The data collection module 501 is configured to collect executable attack codes and generate an attack weapon library based on the executable attack codes.

[0107] The action generation module 502 is configured to input real network information corresponding to a real network environment into an automatic penetration testing decision model to obtain a real attack sub-action, wherein the automatic penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology.

[0108] The penetration testing module 503 is configured to perform automatic penetration testing on the real network environment based on the attack weapon library and the real attack sub-action.

[0109] The embodiment collects executable attack codes and generates an attack weapon library based on the executable attack codes; inputs real network information corresponding to the real network environment into an automatic penetration testing decision model to obtain a real attack sub-action, wherein the automatic penetration testing decision model is a model obtained by training an initial penetration testing model based on deep reinforcement learning technology; and performs automatic penetration testing on the real network environment based on the attack weapon library and the real attack sub-action. Compared with the prior art of generating a penetration testing strategy through manual penetration testing, the embodiment can independently generate a new penetration testing strategy without relying on professional testers, and thus can quickly and accurately perform automatic penetration testing on a network environment, because the automatic penetration testing decision model is obtained by training the initial penetration testing model based on deep reinforcement learning technology, the real attack sub-action is obtained by inputting the real network information corresponding to the real network environment into the automatic penetration testing decision model, and the real network environment is penetrated by combining the real attack sub-action and the executable attack codes in the attack weapon library.

[0110] Based on the first embodiment of the automatic penetration testing device, the second embodiment of the automatic penetration testing device is provided.

[0111] In the embodiment, the data collection module 501 is further configured to perform attack training on a target environment network based on the initial penetration testing model and the attack weapon library, and obtain a feedback reward value returned by the target environment network after being attacked; if the feedback reward value reaches a preset threshold, it is observed whether the performance index of the initial penetration testing model is stably convergent, and if yes, the initial penetration testing model is determined as an automatic penetration testing decision model.

[0112] Further, the data collection module 501 is further configured to repeatedly perform the step of performing attack training on the target environment network based on the initial penetration testing model and the attack weapon library until the performance index of the initial penetration testing model is stably convergent if the performance index of the initial penetration testing model is not stably convergent.

[0113] Further, the data collection module 501 is further configured to reset the target environment network, obtain target initial information corresponding to the target environment network, input the target initial information into the initial penetration testing model to obtain a target attack sub-action, and perform attack on the target environment network based on the attack weapon library and the target attack sub-action.

[0114] Further, the action generation module 502 is further configured to input the real network information corresponding to the real network environment into the automated penetration testing decision model to obtain a sub-action mask of each dimension attack sub-action and a probability distribution of each dimension attack sub-action; determine an effective probability distribution of each dimension attack sub-action based on the sub-action mask and the probability distribution of each dimension attack sub-action, and select a real attack sub-action based on the effective probability distribution.

[0115] Further, the penetration testing module 503 is further configured to assemble a complete attack action based on the attack weapon library and the real attack sub-action, the complete attack action including an attack target, an attack form, and specific attack parameters; perform automated penetration testing on the real network environment according to the complete attack action to obtain updated network environment information corresponding to an updated real network environment and an updated feedback reward value; determine whether a test result of the automated penetration testing meets a preset condition according to the updated network environment information and the updated feedback reward value; and if yes, end the current round of automated penetration testing.

[0116] Further, the penetration testing module 503 is further configured to, if no, perform the following steps until the test result meets the preset condition: input the real network information corresponding to the real network environment into the automated penetration testing decision model to obtain a real attack sub-action; assemble a complete attack action based on the attack weapon library and the real attack sub-action, the complete attack action including an attack target, an attack form, and specific attack parameters; perform automated penetration testing on the real network environment according to the complete attack action to obtain updated network environment information corresponding to an updated real network environment and an updated feedback reward value; and determine whether a test result of the automated penetration testing meets a preset condition according to the updated network environment information and the updated feedback reward value.

[0117] Other embodiments or specific implementations of the automated penetration testing device can refer to the above-mentioned method embodiments, which will not be described here.

[0118] It should be noted that, in this document, the terms “comprising”, “including”, or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or systems that include a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or systems. Without more limitations, the element defined by the statement “comprising a” does not exclude the presence of another identical element in the process, method, article, or system that includes the element.

[0119] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0120] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, an optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0121] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An automated penetration testing method, characterized by, The method comprises the following steps: collecting executable attack codes and generating an attack weapon library based on the executable attack codes; inputting real network information corresponding to a real network environment into an automated penetration testing decision model to obtain real attack sub-actions, the automated penetration testing decision model being a model obtained by training an initial penetration testing model based on deep reinforcement learning technology; automated penetration testing of the real network environment based on the attack weapon library and the real attack sub-actions; the step of automated penetration testing of the real network environment based on the attack weapon library and the real attack sub-actions comprises: assembling a complete attack action based on the attack weapon library and the real attack sub-actions, the complete attack action comprising an attack target, an attack form, and specific attack parameters; automated penetration testing of the real network environment according to the complete attack action to obtain updated network environment information corresponding to an updated real network environment and an updated feedback reward value; determining whether a test result of the automated penetration testing meets a preset condition according to the updated network environment information and the updated feedback reward value; if yes, ending the current round of automated penetration testing.

2. The automated penetration testing method of claim 1, wherein, Before the step of inputting real network information corresponding to a real network environment into an automated penetration testing decision model to obtain real attack sub-actions, the method further comprises: attacking a target environment network based on the initial penetration testing model and the attack weapon library to obtain a feedback reward value returned by the target environment network after being attacked; if the feedback reward value reaches a preset threshold, observing whether a performance index of the initial penetration testing model is stably convergent, and if yes, determining the initial penetration testing model as the automated penetration testing decision model.

3. The automated penetration testing method of claim 2, wherein, After the step of attacking a target environment network based on the initial penetration testing model and the attack weapon library to obtain a feedback reward value returned by the target environment network after being attacked, the method further comprises: if the performance index of the initial penetration testing model is not stably convergent, repeatedly performing the step of attacking the target environment network based on the initial penetration testing model and the attack weapon library until the performance index of the initial penetration testing model is stably convergent.

4. The automated penetration testing method of claim 3, wherein, The step of attacking a target environment network based on the initial penetration testing model and the attack weapon library comprises: resetting the target environment network and obtaining target initial information corresponding to the target environment network; inputting the target initial information into the initial penetration testing model to obtain target attack sub-actions; attacking the target environment network based on the attack weapon library and the target attack sub-actions.

5. The automated penetration testing method of claim 1, wherein, The step of inputting real network information corresponding to a real network environment into an automated penetration testing decision model to obtain real attack sub-actions comprises: inputting real network information corresponding to a real network environment into an automated penetration testing decision model to obtain a sub-action mask of each-dimensional attack sub-actions and a probability distribution of each-dimensional attack sub-actions; An effective probability distribution of the attack sub-actions in each dimension is determined based on the sub-action mask and the probability distribution of the attack sub-actions in each dimension, and a real attack sub-action is selected based on the effective probability distribution.

6. The automated penetration testing method of claim 1, wherein, The step of determining whether the test result of the automated penetration test meets the preset condition according to the updated network environment information and the updated feedback reward value further comprises: If not, the following steps are performed until the test result meets the preset condition: The real attack sub-action is obtained by inputting the real network information corresponding to the real network environment into the automated penetration test decision model. A complete attack action is assembled based on the attack weapon library and the real attack sub-action, and the complete attack action comprises an attack target, an attack form, and specific attack parameters. The real network environment is automatically penetrated according to the complete attack action, and updated network environment information corresponding to the updated real network environment and an updated feedback reward value are obtained. The test result of the automated penetration test is determined whether to meet the preset condition according to the updated network environment information and the updated feedback reward value.

7. An automated permeation testing device, characterized by, The automated penetration test device comprises: A data collection module configured to collect executable attack codes and generate an attack weapon library based on the executable attack codes; An action generation module configured to input real network information corresponding to a real network environment into an automated penetration test decision model to obtain a real attack sub-action, wherein the automated penetration test decision model is a model obtained by training an initial penetration test model based on deep reinforcement learning technology; A penetration test module configured to automatically penetrate the real network environment based on the attack weapon library and the real attack sub-action. The penetration test module is further configured to: Assemble a complete attack action based on the attack weapon library and the real attack sub-action, wherein the complete attack action comprises an attack target, an attack form, and specific attack parameters. Automatically penetrate the real network environment according to the complete attack action, and obtain updated network environment information corresponding to the updated real network environment and an updated feedback reward value. Determine whether the test result of the automated penetration test meets the preset condition according to the updated network environment information and the updated feedback reward value. If yes, end the current round of automated penetration test.

8. An automated penetration testing device, characterized by, The device comprises a memory, a processor, and an automated penetration test program stored on the memory and executable on the processor, wherein the automated penetration test program is configured to implement the steps of the automated penetration test method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium stores an automated penetration test program, and the automated penetration test program is executed by the processor to implement the steps of the automated penetration test method according to any one of claims 1 to 6.

Images