Self-adaptive attack path generation method and device, equipment, medium and program product

Abstract

The invention relates to a self-adaptive attack path generation method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring an attack task set for a target network; in each iteration process, according to the first network environment state of the target network in the current iteration process and the reward result of the target network in the previous iteration process, determining an attack action in the current iteration process; obtaining a reward result after the attack action is executed and a second network environment state of the target network in the current iteration process after the attack action is executed; taking the second network environment state as a first network environment state in a next iteration process, and executing the next iteration process until the attack task is completed; and generating an attack path corresponding to the target network according to the attack action in each iteration process and the corresponding reward result. By adopting the method, a complete and verified attack path can be obtained, so that the comprehensive attack test on the network can be realized.

Description

Technical Field

[0001] This application relates to the field of cybersecurity technology, and in particular to an adaptive attack path generation method, apparatus, device, medium, and program product. Background Technology

[0002] With the rapid development of network information technology, network systems are becoming increasingly complex and large-scale, and network security threats are presenting an unprecedentedly severe situation. Penetration testing has become an indispensable step in testing and improving the defense capabilities of network security systems.

[0003] Mainstream penetration testing methods are primarily divided into two categories: manual penetration testing and automated scanning tools. Manual penetration testing relies heavily on the personal experience of experts, and the comprehensiveness of the test is limited by the expert's skill level. Automated scanning tools use predefined rule bases and vulnerability signature bases to quickly scan target systems and discover known vulnerabilities; however, automated tools typically perform single-point, linear attack tests. Summary of the Invention

[0004] Therefore, it is necessary to provide an adaptive attack path generation method, apparatus, device, medium, and program product that can realize comprehensive attack testing of networks to address the above-mentioned technical problems.

[0005] Firstly, this application provides an adaptive attack path generation method, including:

[0006] Obtain attack tasks configured for the target network;

[0007] In each iteration, the attack action is determined based on the first network environment state of the target network in this iteration and the reward result in the previous iteration.

[0008] Obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action;

[0009] The second network environment state is used as the first network environment state in the next iteration process, and the next iteration process is executed until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network;

[0010] Based on the attack actions and corresponding reward results in each iteration process, the attack path corresponding to the target network is generated.

[0011] In one embodiment, the attack action in the current iteration is determined based on the first network environment state of the target network in the current iteration and the reward result in the previous iteration. This includes: inputting the first network environment state of the target network in the current iteration and the reward result in the previous iteration into the attack strategy generation model to obtain the attack action in the current iteration. The attack strategy generation model is implemented based on a reinforcement learning algorithm.

[0012] In one embodiment, the first network environment state of the target network in the current iteration and the reward result of the previous iteration are input into the attack strategy generation model to obtain the attack action in the current iteration. This includes: determining the loss information of the previous iteration based on the first network environment state in the current iteration, the first network environment state of the previous iteration, and the reward result of the previous iteration; tuning the attack strategy generation model based on the loss information; and inputting the first network environment state of the target network in the current iteration and the reward result of the previous iteration into the tuned attack strategy generation model to obtain the attack action in the current iteration.

[0013] In one embodiment, the first network environment state of the target network in the current iteration and the reward result of the previous iteration are input into the attack strategy generation model to obtain the attack action in the current iteration, including: inputting the first network environment state of the target network in the current iteration and the reward result of the previous iteration into the attack strategy generation model to obtain at least one candidate attack action; and selecting the target attack action whose probability meets the set threshold from the at least one candidate attack action as the attack action in the current iteration.

[0014] In one embodiment, an attack path corresponding to the target network is generated based on the attack actions and corresponding reward results under each iteration process, including: sorting the attack actions with positive reward results according to the execution order of each iteration process; and generating the attack path corresponding to the target network based on the sorted attack actions.

[0015] In one embodiment, obtaining the reward result after executing the attack action includes: obtaining the attack progress status and the evaluation information corresponding to the attack action after executing the attack action; and determining the reward result after executing the attack action based on the attack progress status and the evaluation information.

[0016] Secondly, this application also provides an adaptive attack path generation apparatus, comprising:

[0017] The first acquisition module is used to acquire the attack tasks set for the target network.

[0018] The determination module is used to determine the attack action in each iteration based on the first network environment state of the target network in the current iteration and the reward result in the previous iteration.

[0019] The second acquisition module is used to acquire the reward result after the attack action is executed, as well as the second network environment state of the target network in this iteration process after the attack action is executed;

[0020] The execution module is used to take the second network environment state as the first network environment state in the next iteration process and execute the next iteration process until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network;

[0021] The generation module is used to generate the attack path corresponding to the target network based on the attack actions and corresponding reward results in each iteration process.

[0022] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0023] Obtain attack tasks configured for the target network;

[0024] In each iteration, the attack action is determined based on the first network environment state of the target network in this iteration and the reward result in the previous iteration.

[0025] Obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action;

[0026] The second network environment state is used as the first network environment state in the next iteration process, and the next iteration process is executed until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network;

[0027] Based on the attack actions and corresponding reward results in each iteration process, the attack path corresponding to the target network is generated.

[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0029] Obtain attack tasks configured for the target network;

[0030] In each iteration, the attack action is determined based on the first network environment state of the target network in this iteration and the reward result in the previous iteration.

[0031] Obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action;

[0032] The second network environment state is used as the first network environment state in the next iteration process, and the next iteration process is executed until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network;

[0033] Based on the attack actions and corresponding reward results in each iteration process, the attack path corresponding to the target network is generated.

[0034] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0035] Obtain attack tasks configured for the target network;

[0036] In each iteration, the attack action is determined based on the first network environment state of the target network in this iteration and the reward result in the previous iteration.

[0037] Obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action;

[0038] The second network environment state is used as the first network environment state in the next iteration process, and the next iteration process is executed until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network;

[0039] Based on the attack actions and corresponding reward results in each iteration process, the attack path corresponding to the target network is generated.

[0040] The aforementioned adaptive attack path generation method, apparatus, computer device, computer-readable storage medium, and computer program product, in each iteration, determine the attack action for the current iteration based on the first network environment state of the target network in the current iteration and the reward result of the previous iteration; obtain the reward result after executing the attack action, and the second network environment state of the target network in the current iteration after executing the attack action; use the second network environment state as the first network environment state for the next iteration, and execute the next iteration until the attack task is completed; the above process can obtain the attack actions for each iteration, and by backtracking the above iteration process, the attack actions for each iteration can be chained together to form a complete and verified attack path, thereby enabling comprehensive attack testing of the network. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an application environment diagram of the adaptive attack path generation method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating an adaptive attack path generation method in one embodiment;

[0044] Figure 3 This is a flowchart illustrating the steps for determining the attack actions in this iteration process in one embodiment;

[0045] Figure 4 This is a flowchart illustrating the steps for determining the attack actions in this iteration process in another embodiment;

[0046] Figure 5 This is a flowchart illustrating the steps for determining the attack actions in this iteration process in yet another embodiment;

[0047] Figure 6 This is a flowchart illustrating the adaptive attack path generation method in another embodiment;

[0048] Figure 7 This is a flowchart illustrating the adaptive attack path generation method in yet another embodiment;

[0049] Figure 8 This is a flowchart illustrating the adaptive attack path generation method in yet another embodiment;

[0050] Figure 9 This is a structural block diagram of an adaptive attack path generation device in one embodiment;

[0051] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The adaptive attack path generation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0054] In one exemplary embodiment, such as Figure 2 As shown, an adaptive attack path generation method is provided, which can be applied to... Figure 1 Taking the server in the example, the following S210~S250 are included:

[0055] S210, Obtain attack tasks set for the target network.

[0056] The target network can be understood as the network to be subjected to penetration testing.

[0057] An attack task can be understood as simulating attack behavior to assess the defense capabilities of a target network's network security system. For example, an attack task could be "acquiring root privileges on a server" or "reading the root / flag.txt file."

[0058] An attack task can be understood as an attack task set up to target Internet Protocol (IP) address ranges, domain names, etc. of the target network.

[0059] In one alternative embodiment, the attack task can be manually set and input into the network security system of the target network.

[0060] S220: In each iteration, the attack action for the current iteration is determined based on the first network environment state of the target network in the current iteration and the reward result of the previous iteration.

[0061] The first network environment state can include at least one of the following: port status, currently running services, network (Web) fingerprint information, system attribute information, logged-in user information, and network configuration. Specifically, port status can be understood as an open port; currently running services can be understood as services currently in operation; Web fingerprint information can be understood as specific information exposed by a website in Hypertext Transfer Protocol (HTTP) packets, such as server software, programming language, and framework; system attribute information can be understood as system attribute information, such as system version; logged-in user information can be understood as user information logged into the target network; and network configuration can be understood as the configuration parameters of hardware and / or software in the network.

[0062] In this context, reward outcomes can be understood as the benefits used to incentivize or inhibit the selection of certain behaviors. Reward outcomes can include positive and negative rewards. Positive rewards incentivize the selection of certain behaviors, while negative rewards inhibit the selection of certain behaviors.

[0063] The reward result of the previous iteration can be understood as the total reward accumulated from the first iteration to the previous iteration.

[0064] In this context, an attack action can be understood as the action performed to launch an attack on a target network.

[0065] In one optional embodiment, during the first iteration, the initial network environment state under the first iteration can be obtained as the first network environment state under the current iteration. In subsequent iterations, the second network environment state after the attack action was performed in the previous iteration can be used as the first network environment state under the local iteration.

[0066] In one alternative embodiment, the reward result of the previous iteration can be the reward result obtained after performing the attack action in the previous iteration.

[0067] In one optional embodiment, when this iteration is the first iteration, the attack action can be derived from the initial network environment state of the target network. When this iteration is not the first iteration, the attack action can be determined based on the first network environment state of the target network in this iteration and the reward result of the previous iteration.

[0068] In one alternative embodiment, the action execution module can be used to parse the description information of the attack action and call the corresponding tools or construct network data packets to perform the attack action on the target network in the target environment.

[0069] S230: Obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action.

[0070] In one optional embodiment, obtaining the reward result after executing the attack action can be performed according to the following steps: obtaining the attack progress status and the evaluation information corresponding to the attack action after executing the attack action; determining the reward result after executing the attack action based on the attack progress status and the evaluation information.

[0071] The attack progress status can be understood as a Boolean value indicating whether progress has been made, or as the degree of attack progress from the second network environment state in the previous iteration to the second network environment state in the current iteration. The Boolean value can be true or false; the Boolean value for whether progress has been made can be either true or false. Whether progress has been made can be determined by whether new credentials or permissions have been obtained.

[0072] In one alternative embodiment, the reward result after performing the attack action can be determined according to the following formula:

[0073]

[0074] in, Indicates time The reward results; A Boolean value indicating whether progress has been made; The weight corresponding to the Boolean value indicating whether progress has been made; Indicates time The evaluation information is obtained by evaluating the network environment status under the given conditions; This indicates the weight corresponding to the evaluation information; Indicates the execution of an attack action. The cost of resources consumed, which may include time costs or computational resource costs; Indicates the execution of an attack action. The weight corresponding to the cost of the resources consumed.

[0075] In one alternative embodiment, the reward result after performing the attack action can be determined according to the following formula:

[0076]

[0077] in, Indicates time The reward results; Indicates from time The network environment status until time The extent of the attack's progress in the network environment under the given conditions; Weights representing the degree of attack progress; Indicates an attack action The evaluation information obtained by evaluating the concealment; This indicates the weight corresponding to the evaluation information; Indicates quantization time The permission level for the network environment status; This indicates the weight corresponding to the permission level.

[0078] In one optional embodiment, the verification module can be used to detect changes in the network environment state after the attack action is executed, and obtain the verification result; then, the reward result is determined based on the verification result.

[0079] The verification results are used to indicate whether the attack was successful, such as whether a new shell was obtained, whether sensitive data was returned, and whether the port status changed.

[0080] In one optional embodiment, a network environment state awareness module can be used to perform non-intrusive information collection on the target network to obtain a second network environment state during the current iteration. For example, port status can be obtained through a port scanning module; currently running services can be obtained through a service identification module; and web fingerprint information can be obtained through a web fingerprint detection module.

[0081] In one alternative embodiment, the information received by the network environment state awareness module can be structured to obtain a second network environment state in vector form.

[0082] S240, the second network environment state is used as the first network environment state in the next iteration process, and the next iteration process is executed until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network.

[0083] S250 generates the attack path corresponding to the target network based on the attack actions and corresponding reward results in each iteration process.

[0084] The attack path can be understood as a complete chain of attack actions, from the initial attack action to the completion of the attack task.

[0085] In one optional embodiment, attack actions with positive reward results can be sorted according to the execution order of each iteration process; then, attack paths corresponding to the target network can be generated based on the sorted attack actions.

[0086] In this embodiment, by generating the attack path corresponding to the target network based on the sorted attack actions, a complete and comprehensive attack path can be obtained.

[0087] In the aforementioned adaptive attack path generation method, in each iteration, the attack action is determined based on the first network environment state of the target network in this iteration and the reward result of the previous iteration. The reward result after executing the attack action and the second network environment state of the target network in this iteration are obtained. The second network environment state is used as the first network environment state in the next iteration, and the next iteration is executed until the attack task is completed. The above process can obtain the attack actions in each iteration. By backtracking the above iteration process, the attack actions in each iteration can be linked together to form a complete and verified attack path, thereby enabling comprehensive attack testing of the network.

[0088] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the steps for determining the attack action in the current iteration process are refined.

[0089] like Figure 3 As shown, the steps to determine the attack actions in this iteration process include:

[0090] S310: Input the first network environment state of the target network in this iteration process and the reward result of the previous iteration process into the attack strategy generation model to obtain the attack action in this iteration process.

[0091] The attack strategy generation model can be implemented based on reinforcement learning algorithms.

[0092] The attack strategy generation model can be understood as a model used to generate attack actions in the current iteration process based on the first network environment state of the target network in the current iteration process and the reward results in the previous iteration process.

[0093] Attack strategy generation models can be implemented based on reinforcement learning algorithms.

[0094] Optionally, the attack strategy generation model can be a Large Language Model (LLM).

[0095] In one optional embodiment, the first network environment state of the target network in the current iteration and the reward result of the previous iteration can be input into the attack strategy generation model to obtain the attack strategy in the current iteration.

[0096] The attack strategy may include attack actions.

[0097] In one alternative embodiment, the attack strategy generation model can leverage its built-in capabilities regarding vulnerabilities, attack tools, tactics, techniques, and procedures (TTPs) to generate attack strategies, such as "using an exploit script to attack port 445 using the ms17-010 vulnerability" or "attempting brute-force attacks on the login page for the first 100 weak passwords."

[0098] In this embodiment, by utilizing an attack strategy generation model based on reinforcement learning algorithms, the decision direction of attack actions can be adjusted, thereby enabling the generation of attack actions that can bring greater rewards in subsequent decisions.

[0099] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the steps for determining the attack action in the current iteration process are refined.

[0100] like Figure 4 As shown, the steps to determine the attack actions in this iteration process include:

[0101] S410. Based on the first network environment state in the current iteration, the first network environment state in the previous iteration, and the reward result in the previous iteration, determine the loss information in the previous iteration.

[0102] In one optional embodiment, the loss information of the previous iteration can be determined based on the difference between the first network environment state in the current iteration and the first network environment state in the previous iteration, as well as the reward result in the previous iteration.

[0103] S420 adjusts the parameters of the attack strategy generation model based on the loss information.

[0104] S430: Input the first network environment state of the target network in this iteration and the reward result of the previous iteration into the parameter-tuned attack strategy generation model to obtain the attack action in this iteration.

[0105] Among them, the parameter-tuned attack strategy generation model can obtain better attack actions based on the first network environment state of the target network in this iteration process and the reward results in the previous iteration process.

[0106] In this embodiment, by adjusting the parameters of the attack strategy generation model based on the first network environment state in the current iteration, the first network environment state in the previous iteration, and the reward result in the previous iteration, the adjusted attack strategy generation model can learn from successful and unsuccessful experiences and obtain more optimized attack actions.

[0107] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the steps for determining the attack action in the current iteration process are refined.

[0108] like Figure 5 As shown, the steps to determine the attack actions in this iteration process include:

[0109] S510: Input the first network environment state of the target network in this iteration and the reward result in the previous iteration into the attack strategy generation model to obtain at least one candidate attack action.

[0110] S520: From at least one candidate attack action, select the target attack action whose probability meets the set threshold as the attack action in this iteration process.

[0111] The threshold can be a preset threshold.

[0112] In one optional embodiment, the target attack action with the highest probability value can be selected from at least one candidate attack action as the attack action in the current iteration process.

[0113] In this embodiment, by selecting the target attack action whose probability meets the set threshold as the attack action in this iteration process, a better attack action can be obtained.

[0114] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the adaptive attack path generation method is described in detail.

[0115] like Figure 6 The adaptive attack path generation method shown includes:

[0116] S601, obtain the attack task set for the target network, as well as the initial network environment status of the target network.

[0117] In one alternative embodiment, the operator can input network information of the target network and the attack task. The network information of the target network may include at least one of the following: IP address ranges and domain names.

[0118] S602, during the first iteration, the initial network environment state of the target network is input into the attack strategy generation model to obtain at least one candidate attack action.

[0119] S603: Select the target attack action with the highest probability from at least one candidate attack action as the attack action in this iteration process.

[0120] S604, obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action, and use the second network environment state as the first network environment state in the next iteration process.

[0121] In one optional embodiment, the attack progress status and evaluation information corresponding to the attack action are obtained after the attack action is executed; based on the attack progress status and evaluation information, the reward result after the attack action is executed is determined.

[0122] S605, in non-first iterations, the second network environment state under the previous iteration is used as the first network environment state under the current iteration. Based on the first network environment state, the first network environment state under the previous iteration, and the reward result under the previous iteration, the loss information under the previous iteration is determined.

[0123] S606, based on the loss information, adjusts the parameters of the attack strategy generation model.

[0124] S607. Input the first network environment state of the target network in this iteration process and the reward result in the previous iteration process into the parameter-tuned attack strategy generation model to obtain at least one candidate attack action.

[0125] S608: Select the target attack action with the highest probability from at least one candidate attack action as the attack action in this iteration process.

[0126] S609, obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action.

[0127] S610, determine whether the attack task has been completed; if not, execute the step of S605, which takes the second network environment state as the first network environment state in the next iteration process; if yes, execute S611.

[0128] S611: Sort the attack actions with positive rewards according to the execution order of each iteration process.

[0129] S612 generates the attack path corresponding to the target network based on the sorted attack actions.

[0130] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the adaptive attack path generation method is described in detail.

[0131] This adaptive attack path generation method can be applied to scenarios where financial services companies want to conduct a comprehensive security assessment of their newly launched online trading platforms to ensure the security of customer data. The platform's backend is built using Hypertext Preprocessor (PHP) and Apache web server software, and integrates an open-source shopping cart system. The company wants to simulate how hackers exploit vulnerabilities in the web application to gradually penetrate from the external network and ultimately gain control of the server, in order to test the effectiveness of its defense-in-depth system. This embodiment demonstrates the application of the adaptive attack path generation method in an automated web application penetration testing scenario. The target is this online shopping mall system, and the ultimate goal is to gain control of the web server.

[0132] like Figure 7 The adaptive attack path generation method is shown below. First, the operator sets the attack target to the target IP address of the online shopping mall system, with the ultimate goal of "obtaining a Web Shell and executing the `whoami` command to display the username of the currently logged-in user." After system startup, the environment awareness module first collects network environment information from the online shopping mall system, which then returns the corresponding information. Specifically, a NetworkMapper (Nmap) scan is performed on the target IP address to obtain its open ports 80 (HTTP) and 443 (Hypertext Transfer Protocol Secure, HTTPS). Next, a Web fingerprinting tool is used to detect that the website's backend language is PHP, the web server is Apache, and it runs an open-source e-commerce system called "EasyCart." This information is then integrated to form the initial network environment state. and the initial network environment state The input is fed into the attack strategy generation model implemented using LLM. The attack strategy generation model receives... Subsequently, vulnerability information related to "EasyCart", "PHP", and "Apache" was retrieved from the database corresponding to the online marketplace system. This was based on the initial network environment status. Based on the retrieved vulnerability information, a specific version of the open-source e-commerce system "EasyCart" was identified as having a Structured Query Language (SQL) injection vulnerability located in the Identification (ID) parameter of the product search interface search.php. Based on this vulnerability, the attack strategy generation model can generate the first attack action. "Perform a boolean-based blind SQL injection on the search.php?id= parameter to confirm the existence of the vulnerability."

[0133] The action execution module received Then, a pre-built Python script is invoked. This script uses a request library to construct HTTP requests, such as "... / search.php?id=1' AND 1=1" and "... / search.php?id=1' AND 1=2--", and executes an SQL query. The result verification and feedback module confirms the existence of the SQL injection vulnerability by comparing the page results returned by the two requests. Because this is a crucial breakthrough, the system can award a high positive reward. The network environment status has been updated to... The report included the new information that "an SQL injection vulnerability exists."

[0134] Next, Input is fed into the attack strategy generation model. The attack strategy generation model is based on the new network environment state. Generate attack actions "Exploiting an SQL injection vulnerability, the administrator username and password hash were read from the database." The action execution module called the SQLMap tool, passing in the corresponding parameters, to execute a malicious query. After a period of probing, SQLMap successfully extracted the password hash of the administrator "admin" from the users table. The result verification module confirmed the hash value was obtained, awarding another significant positive reward. And update the network environment status to .

[0135] In network environment status The attack strategy generation model is based on The attack strategy can be derived from cracking the password and generating corresponding attack actions. "The hash obtained was cracked using the John the Ripper tool and a common password dictionary." The action execution module invoked the cracking tool and successfully cracked the plaintext password "password123". This was a significant breakthrough, and the system awarded a higher reward. Update the network environment status to At this point, the agent has administrator privileges. The attack strategy generation model can deduce that logging into the backend with administrator privileges usually allows access to file upload points, a common method for obtaining a web shell. Therefore, it generates attack actions. The task was: "Log in to the website backend, find the file upload function, and attempt to upload a PHP web shell." The action execution module used the Selenium library to simulate browser behavior, automating the process of logging in, finding the upload point, and uploading a web shell file named "shell.php". The result verification module confirmed the successful upload by accessing the uploaded file URL (i.e., ... / uploads / shell.php) and checking if the returned content met expectations. This almost achieved the final goal, thus earning a significant positive reward. .

[0136] Finally, the attack strategy generation model generates the attack actions to achieve the final goal. The attack objective was: "Execute the 'whoami' command to display the username of the currently logged-in user via Web Shell." The action execution module sent a data packet containing a request to execute the command to shell.php. The result verification module captured the returned result, www-data, which matched the expected output of "Execute the 'whoami' command to display the username of the currently logged-in user via Web Shell." The attack objective was achieved, and the system awarded the final, highest reward. The process is then terminated. The reward for each iteration can be obtained using the following formula:

[0137]

[0138] in, Indicates time The reward results; A Boolean value indicating whether progress has been made; The weight corresponding to the Boolean value indicating whether progress has been made; Indicates time The evaluation information is obtained by evaluating the network environment status under the given conditions; This indicates the weight corresponding to the evaluation information; Indicates the execution of an attack action. The cost of resources consumed, which may include time costs or computational resource costs; Indicates the execution of an attack action. The weight corresponding to the cost of the resources consumed.

[0139] Ultimately, the system will reward the corresponding attack action sequence with positive feedback. , , , , These are linked together to generate a detailed attack report, clearly showing the complete path from SQL injection to the final remote code execution.

[0140] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment in which the adaptive attack path generation method is described in detail.

[0141] This adaptive attack path generation method can be applied to simulate scenarios where an enterprise intranet is attacked by an Advanced Persistent Threat (APT). The simulation starts with a regular employee's personal computer (PC) that has been compromised and has limited access to some network resources. The ultimate goal is to gain administrator privileges on the core of the entire enterprise network—the Domain Control (DC), which is usually one of the ultimate targets of APT attacks.

[0142] like Figure 8 The adaptive attack path generation method is shown. Assume an attacker has compromised an ordinary employee's office computer PC-A via spear-phishing emails, with the goal of gaining administrator privileges on the domain controller.

[0143] First, network environment information is collected from the office computers. The office computers return network environment information, such as PC-A's system information (e.g., system version and logged-in users) and network configuration. Based on this network environment information, an initial network environment state is generated. The attack mission is set to "execute arbitrary commands on the domain controller DC". The attack strategy generation model receives... Then, the first attack action generated. The objective is: "To collect local information on PC-A, including network neighbors, local users, running processes, etc." The action execution module executes commands such as "net view," "whoami / all," and "tasklist" in the PC-A's shell. The result verification and feedback module parses the command output, discovers that PC-A is located in the CORP.LOCAL domain, and identifies the file server and domain controller. The network environment status is updated to... And give a small reward for information collection. .

[0144] In state The attack strategy generation model first attempts the most direct privilege escalation method to generate attack actions. "Run the Mimikatz tool on PC-A to capture plaintext passwords or hash values." The action execution module attempts to load the Mimikatz tool but fails due to insufficient permissions. The result verification module detects the failure and issues a negative reward. This prompted the attack strategy generation model to abandon this approach and explore other possibilities. Instead, the attack strategy generation model analyzed newly discovered hosts to generate attack actions. "Scan for common service vulnerabilities on the file server, especially the Server Message Block (SMB) service." The action execution module calls the vulnerability scanning script of the Nmap tool to scan port 445 of the file server and reports the presence of the MS17-010 vulnerability. This is a very valuable discovery, and the system awards a substantial positive reward. The network environment status has been updated to... .

[0145] based on The attack strategy generation model immediately generated the most aggressive actions. "Attacked a file server using the ms17_010_eternalblue module in the Metasploit framework." The action execution module was configured and launched, successfully gaining system privileges on the file server. This was a decisive breakthrough, and the system awarded a substantial positive reward. and the network environment status Updated to "File server under control".

[0146] After gaining control of the file server, the primary goal of the attack strategy generation model is to find a stepping stone to the domain controller on this higher-privilege server. It generates attack actions. "In the file server's system session, run the Mimikatz tool again to capture all user credentials in memory." Due to sufficiently high system privileges, the Mimikatz tool ran successfully this time, dumping the credentials of recently logged-in users from memory, including a Kerberos ticket for a domain administrator, CORP\DomainAdmin. The verification module confirmed the acquisition of the domain administrator's credentials, the key to the final goal, and the system awarded a very high reward. The network environment status has been updated to... .

[0147] Finally, the attack strategy generation model generates the final attack actions. "Using the obtained domain administrator credentials, remotely log in to the domain controller via the PsExec tool." The action execution module calls the PsExec tool and successfully establishes a shell session with the highest privileges on the domain controller using the domain administrator CORP\DomainAdmin credentials. The result verification module confirms that the returned result is CORP\DomainAdmin by executing the "whoami" command on the domain controller, indicating that administrator privileges on the domain controller have been successfully obtained. The attack objective has been fully achieved, and the system awards the final and highest reward. And terminate the entire attack process.

[0148] In this embodiment, the reward function design for reinforcement learning places greater emphasis on long-term policy value, and can be expressed as:

[0149]

[0150] in, Indicates time The reward results; Indicates from time The network environment status until time The extent of the attack's progress in the network environment under the given conditions; Weights representing the degree of attack progress; Indicates an attack action The evaluation information obtained by evaluating the concealment; This indicates the weight corresponding to the evaluation information; Indicates quantization time The permission level for the network environment status; This indicates the weight corresponding to the permission level.

[0151] Ultimately, the system can generate a complete internal network lateral movement attack report, detailing the entire attack path from ordinary user privileges to domain administrator privileges: PC-A information gathering → file server vulnerability scanning → MS17-010 vulnerability exploitation → obtaining system privileges → Mimikatz credential theft → remote login to the domain controller.

[0152] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0153] Based on the same inventive concept, this application also provides an adaptive attack path generation apparatus for implementing the adaptive attack path generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the adaptive attack path generation apparatus provided below can be found in the limitations of the adaptive attack path generation method described above, and will not be repeated here.

[0154] In one exemplary embodiment, such as Figure 9 As shown, an adaptive attack path generation device is provided, comprising: a first acquisition module 910, a determination module 920, a second acquisition module 930, an execution module 940, and a generation module 950, wherein:

[0155] The first acquisition module 910 is used to acquire the attack tasks set for the target network;

[0156] The determination module 920 is used to determine the attack action in each iteration based on the first network environment state of the target network in the current iteration and the reward result in the previous iteration.

[0157] The second acquisition module 930 is used to acquire the reward result after the attack action is executed, and the second network environment state of the target network in this iteration process after the attack action is executed;

[0158] The execution module 940 is used to take the second network environment state as the first network environment state in the next iteration process and execute the next iteration process until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network;

[0159] The generation module 950 is used to generate the attack path corresponding to the target network based on the attack actions and corresponding reward results in each iteration process.

[0160] In one embodiment, the determining module 920 is specifically used to: input the first network environment state of the target network in the current iteration process and the reward result of the previous iteration process into the attack strategy generation model to obtain the attack action in the current iteration process; the attack strategy generation model is implemented based on the reinforcement learning algorithm.

[0161] In one embodiment, the determining module 920 is specifically used to: determine the loss information of the previous iteration based on the first network environment state of the current iteration, the first network environment state of the previous iteration, and the reward result of the previous iteration; adjust the parameters of the attack strategy generation model based on the loss information; and input the first network environment state of the target network in the current iteration and the reward result of the previous iteration into the adjusted attack strategy generation model to obtain the attack action in the current iteration.

[0162] In one embodiment, the determining module 920 is specifically used to: input the first network environment state of the target network in the current iteration process and the reward result of the previous iteration process into the attack strategy generation model to obtain at least one candidate attack action; and select the target attack action whose probability meets the set threshold from the at least one candidate attack action as the attack action in the current iteration process.

[0163] In one embodiment, the generation module 950 is specifically used to sort the attack actions with positive reward results according to the execution order of each iteration process; and generate the attack path corresponding to the target network according to the sorted attack actions.

[0164] In one embodiment, the second acquisition module 930 is specifically used to acquire the attack progress status and the evaluation information corresponding to the attack action after the attack action is executed; and to determine the reward result after the attack action is executed based on the attack progress status and the evaluation information.

[0165] Each module in the aforementioned adaptive attack path generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0166] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores relevant data for the adaptive attack path generation method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an adaptive attack path generation method.

[0167] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the adaptive attack path generation method provided in any of the above embodiments.

[0169] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the adaptive attack path generation method provided in any of the above embodiments.

[0170] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the adaptive attack path generation method provided in any of the above embodiments.

[0171] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0172] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0173] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An adaptive attack path generation method, characterized in that, The method includes: Obtain attack tasks configured for the target network; In each iteration, the attack action is determined based on the first network environment state of the target network in this iteration and the reward result in the previous iteration. Obtain the reward result after executing the attack action, and the second network environment state of the target network in this iteration process after executing the attack action; The second network environment state is used as the first network environment state in the next iteration process, and the next iteration process is executed until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network; Based on the attack actions and corresponding reward results in each iteration process, the attack path corresponding to the target network is generated.

2. The method according to claim 1, characterized in that, The step of determining the attack action in the current iteration based on the first network environment state of the target network in this iteration and the reward result in the previous iteration includes: The first network environment state of the target network in this iteration and the reward result of the previous iteration are input into the attack strategy generation model to obtain the attack action in this iteration; the attack strategy generation model is implemented based on the reinforcement learning algorithm.

3. The method according to claim 2, characterized in that, The step of inputting the first network environment state of the target network in the current iteration and the reward result of the previous iteration into the attack strategy generation model to obtain the attack actions in the current iteration includes: Based on the first network environment state in this iteration, the first network environment state in the previous iteration, and the reward result in the previous iteration, determine the loss information in the previous iteration. Based on the loss information, the parameters of the attack strategy generation model are tuned; The target network's first network environment state in this iteration and the reward result in the previous iteration are input into the parameter-tuned attack strategy generation model to obtain the attack action in this iteration.

4. The method according to claim 2 or 3, characterized in that, The step of inputting the first network environment state of the target network in the current iteration and the reward result of the previous iteration into the attack strategy generation model to obtain the attack actions in the current iteration includes: The first network environment state of the target network in this iteration and the reward result in the previous iteration are input into the attack strategy generation model to obtain at least one candidate attack action; From the at least one candidate attack action, select the target attack action whose probability meets the set threshold as the attack action in this iteration process.

5. The method according to claim 1, characterized in that, The step of generating the attack path corresponding to the target network based on the attack actions and corresponding reward results in each iteration process includes: According to the execution order of each iteration process, attack actions with positive reward results are sorted. Based on the sorted attack actions, an attack path corresponding to the target network is generated.

6. The method according to claim 1, characterized in that, The process of obtaining the reward result after executing the attack action includes: Obtain the attack progress status and evaluation information corresponding to the attack action after the attack action is executed; Based on the attack progress and the evaluation information, the reward result after executing the attack action is determined.

7. An adaptive attack path generation device, characterized in that, The device includes: The first acquisition module is used to acquire the attack tasks set for the target network. The determination module is used to determine the attack action in each iteration based on the first network environment state of the target network in the current iteration and the reward result in the previous iteration. The second acquisition module is used to acquire the reward result after the attack action is executed, and the second network environment state of the target network in this iteration process after the attack action is executed. An execution module is used to take the second network environment state as the first network environment state in the next iteration process and execute the next iteration process until the attack task is completed; wherein, the attack action in the first iteration process is obtained based on the initial network environment state of the target network; The generation module is used to generate the attack path corresponding to the target network based on the attack actions and the corresponding reward results in each iteration process.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images