A multi-agent automated penetration testing system integrating large models and reinforcement learning

Abstract

The present invention discloses a multi-agent automated penetration testing system integrating a large model and reinforcement learning, which relates to the field of network security technology, including: starting a hybrid multi-agent automated penetration testing process combined with a large model by inputting a URL; multi-agents jointly acting to comprehensively collect and classify website information; multi-agents perform vulnerability mining in parallel, and feedback the results for deep reinforcement learning optimization; multi-agents formulate specific utilization plans for the mined vulnerabilities; multi-agents collaboratively verify the effectiveness of potential vulnerabilities, recycle the results and use the deep learning feedback mechanism to gradually improve the accuracy of decision-making; multi-agents generate security reports based on module results, and feedback to related agents to enhance deep learning performance and improve decision-making effectiveness. The present invention integrates large model analysis and reasoning capabilities, realizes multi-agent collaborative scheduling, performs automated security scanning on target websites, and effectively detects and evaluates potential security vulnerabilities.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a multi-agent automated penetration testing system integrating a large model and reinforcement learning. Background Art

[0002] In the field of network security, penetration testing is a key strategy for evaluating the security of information systems. At present, the widely used penetration testing methods rely heavily on manual operations, which not only reduces the efficiency of the testing process, but also may lead to the omission of potential security threats. Furthermore, the existing penetration testing tools and methods show obvious limitations when dealing with complex network structures and diversified attack methods. Specifically, these tools and methods are insufficient in the following two aspects: first, they lack the ability to quickly scan and deeply analyze vulnerabilities for target systems of multiple service types; second, they lack the necessary specificity and accuracy for the detection and exploitation of security vulnerabilities in specific industries. In addition, the existing open source vulnerability scanning tools have limited applicability and complex operations, and automated penetration testing tools are often limited to detecting specific types of vulnerabilities, which limits the breadth of their detection range.

[0003] In recent years, the rapid development of artificial intelligence technology has brought changes to many industries, including the security field. However, due to the complexity of security operations involving a wide range of knowledge systems, and as security threats continue to evolve, large models are limited in their processing capabilities when dealing with complex data and scenarios, resulting in insufficient accuracy of results, as well as deficiencies in knowledge acquisition and real-time learning of new data.

[0004] According to existing literature, penetration testing methods are no longer limited to traditional manual penetration methods. At present, automated tool penetration and penetration technology based on big data models are gradually developing. Specifically, representative studies include rule-based and model-based penetration testing methods (see Chen Ke, Lu Hui, Fang Binxing, Sun Yanbin, Su Shen, Tian Zhihong. A review of research on automated penetration testing technology. Journal of Software, 2024, 35(5): 2268-2288), as well as an iterative attack method for an automated penetration testing system based on artificial intelligence (AI) (see Beijing Moyun Technology Co., Ltd. An iterative attack method for an automated penetration testing system based on AI. CN 111581645 A[P]. 2020-08-25) and an AI-based website penetration testing method (see Jiangsu Hengtong Industrial Control Security Research Institute Co., Ltd. A website penetration testing method based on AI. CN 110221977 B[P]. 2023-04-14). These studies demonstrate the research on automation and intelligence of penetration testing technology, but they all have the shortcomings of not deeply exploring the complexity of integrating large models with deep reinforcement learning and not mentioning the collaborative work of multiple agents. Summary of the invention

[0005] The present invention aims to solve the problems of low efficiency of penetration testing in manual operation and automated tool application, insufficient ability to cope with complex network structures and diversified attack methods, and insufficient specificity and accuracy of vulnerability detection in the prior art, while overcoming the limitations of artificial intelligence in real-time updating and optimizing decision-making. To this end, a hybrid multi-agent automated penetration testing system integrating large models and deep reinforcement learning is proposed. It adopts a multi-agent system driven by a large-scale model to realize collaborative parallel vulnerability mining and utilization technology. At the same time, it focuses on its in-depth application and optimization in the field of network security based on the intelligent feedback learning mechanism of the large-scale model, which can significantly improve the efficiency, accuracy and comprehensiveness of penetration testing.

[0006] The technical solution of the present invention is as follows.

[0007] A multi-agent automated penetration testing system that integrates large models and reinforcement learning, including:

[0008] A penetration test initiation module, which is used to receive a valid URL of an accessible website as input and initiate a hybrid multi-agent automated penetration test process combined with a large model;

[0009] The information collection module is used to generate scheduling scripts through tool scheduling agents, schedule localized tool libraries, collect website information, and process information through application layer, network layer and transport layer through the set agents, identify relevant service information, filter out duplicate data, and extract information at each level;

[0010] The vulnerability mining module is used to integrate the information collected by the information collection module at the application layer, network layer and transport layer, and combines it with the localized vulnerability scanning tool based on YAML template rules. It uses the Web application framework vulnerability mining agent group, the middleware vulnerability mining agent group and the system transport layer service vulnerability mining agent group to perform vulnerability scanning and identification. The vulnerability mining text processing scheduling agent is responsible for coordinating the activities of different service agents to form a vulnerability mining report.

[0011] The vulnerability solution generation module is used to integrate the vulnerability information identified by the vulnerability mining module and the localized deployment scanning results based on the vulnerability scanning tool. The vulnerability mining information processing scheduling agent schedules the corresponding professional vulnerability solution generation agent group, and customizes the corresponding vulnerability exploitable solution for each specific vulnerability in combination with the vulnerability library information.

[0012] The vulnerability verification module is used to operate on multiple different vulnerability information based on the generated vulnerability exploitation schemes. The vulnerability verification scheduling agent is responsible for coordinating the professional vulnerability verification agent group including the Web application framework, middleware and transport layer to achieve concurrent operation. Each vulnerability information generates an executable vulnerability exploitation script, dispatches local resources to execute the script and analyze the results, and generates vulnerability information and exploitation scripts.

[0013] The report generation module is used to integrate the results generated by the information collection module, vulnerability mining module, vulnerability solution generation module and vulnerability verification module, provide a variety of model options to optimize data processing, perform filtering, exclude junk and duplicate information functions, generate a security assessment report based on a predetermined security report template, and convert the data processed by the information collection module, vulnerability mining module, vulnerability solution generation module and vulnerability verification module into training samples to enhance the performance of each intelligent agent and realize a feedback learning mechanism.

[0014] Preferably, the information collection module includes:

[0015] A large model selection unit is used to select and process a large model to carry out information collection and reasoning tasks;

[0016] The collection decision acquisition unit is composed of a tool scheduling agent that designs the information collection process based on the predetermined workflow and tool library information, decides which tool combination to use to achieve more comprehensive information collection, and obtains the collection decision;

[0017] An information collection text generation unit, used for using the collection decision to schedule local tool resources to capture information and form an information collection text;

[0018] A deduplication unit, used for filtering out duplicate data using an information filtering agent;

[0019] An intermediate data set text generation unit is used to use an information collection scheduling agent to schedule different agents to divide the collected data, and the processing agents of the application layer, the network layer and the transport layer process the collected information to form an intermediate data set text;

[0020] The user decision unit is used to provide the generated intermediate data set text to the user for reading and decision-making, to decide whether to perform vulnerability mining operations, whether to re-collect information or whether to exit the system.

[0021] Preferably, the vulnerability mining module includes:

[0022] The information processing and scheduling unit is used to classify the collected results based on the intermediate data set text collected by the information collection module and the results generated by the local vulnerability scanning tool according to the YAML format matching rules. The information collection document processing agent classifies the collected results and distributes them to the corresponding dedicated agents for analysis;

[0023] A scanning task execution unit is used to use the information collection document processing scheduling agent to generate and schedule the corresponding framework vulnerability scanning tool, execute the scanning task, and collect the scanning results;

[0024] The agent grouping and vulnerability library configuration unit is used to divide the scheduling information collection document processing scheduling agent into a Web application framework vulnerability mining agent group, a Web middleware vulnerability mining agent group, and a system transport layer service vulnerability mining agent group, and correspond to the corresponding vulnerability library information to build an information knowledge base;

[0025] The service information-driven vulnerability mining unit is used to use the vulnerability mining text processing scheduling agent to coordinate the activities of the Web application framework vulnerability mining agent group, the middleware vulnerability mining agent group, and the system transport layer service vulnerability mining agent group, and then coordinate the service type-specific agents corresponding to the degree to perform vulnerability mining tasks for targeted services and form a vulnerability mining report.

[0026] Preferably, the vulnerability solution generation module includes:

[0027] The vulnerability data processing and scheduling unit is used to schedule the vulnerability scanning results of the localization tool based on the report generated by the vulnerability mining module and the vulnerability mining information processing agent based on the POC, classify the result text and assign it to the corresponding service type dedicated agent for processing;

[0028] The data reasoning model selection unit is used to select a large model suitable for vulnerability data reasoning to support the subsequent agent reasoning process;

[0029] The agent grouping and solution generation unit is used to divide the vulnerability exploitation solution generation module into an agent group based on Web application vulnerability exploitation solution generation, an agent group based on middleware vulnerability exploitation solution generation, and an agent group based on system transmission vulnerability exploitation solution generation according to different vulnerability types. Each agent group performs corresponding reasoning tasks and generates vulnerability exploitation solutions.

[0030] Preferably, the vulnerability verification module includes:

[0031] A vulnerability exploitation scheme scheduling unit, used to use the vulnerability exploitation scheme information to schedule the agent allocation and schedule the corresponding service type-specific agent, and select a large model to execute the reasoning process of the vulnerability exploitation script;

[0032] The agent grouping and task allocation unit is used to divide the vulnerability verification module into a Web application service framework vulnerability exploitation script generation agent group, a middleware vulnerability exploitation script generation agent group, and a system transport layer vulnerability exploitation script generation agent group, which are responsible for the corresponding vulnerability mining tasks;

[0033] The vulnerability exploitation script generation and testing unit is used to schedule the corresponding service type-specific intelligent agent to generate vulnerability exploitation scripts and schedule local resources for testing; if the test is successful, vulnerability verification data is generated; otherwise, it is determined whether it is a coding problem. If it is a coding problem, an error log training sample is generated and the vulnerability exploitation solution scheduling unit is repeatedly executed. If it is not a coding problem, the vulnerability information is not recorded.

[0034] Preferably, the report generating module comprises:

[0035] A vulnerability verification data processing unit, used for selecting a suitable large model to process the vulnerability verification data generated by the vulnerability verification module;

[0036] A report compilation unit for generating a readable and safe report using the selected large model;

[0037] The data recording and sample construction unit is used to record the data generated by all operations in the process into the database to form a complete agent training sample set.

[0038] Preferably, the multi-agent automated penetration testing system integrating large models and reinforcement learning also includes an update module for regularly updating the agent vulnerability library and tool library.

[0039] Preferably, the update module includes:

[0040] A vulnerability library information supplement unit is used to regularly review and update the vulnerability library information of the intelligent agent;

[0041] The tool library expansion unit is used to regularly identify and evaluate new security testing tools and incorporate them into the tool library.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention provides a multi-agent automated penetration testing system that integrates large models and reinforcement learning. By integrating the analytical reasoning capabilities of a trillion-dollar model, it realizes the collaborative scheduling of multiple agents, thereby performing automated security scans on target websites, effectively detecting and evaluating potential security vulnerabilities, and significantly improving the efficiency, accuracy, and comprehensiveness of penetration testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A processing schematic diagram of a multi-agent automated penetration testing system integrating a large model and reinforcement learning according to an embodiment of the present invention;

[0046] Figure 2 This is a processing diagram of an information collection module according to an embodiment of the present invention;

[0047] Figure 3 A processing schematic diagram of a vulnerability mining module according to an embodiment of the present invention;

[0048] Figure 4 A processing schematic diagram of a vulnerability solution generation module according to an embodiment of the present invention;

[0049] Figure 5 A processing diagram of a vulnerability verification module according to an embodiment of the present invention;

[0050] Figure 6 A processing schematic diagram of a report generation module according to an embodiment of the present invention;

[0051] Figure 7 The present invention is a flowchart of a multi-agent automated penetration testing method that integrates a large model and reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] See also Figure 1 As shown, the present invention provides a multi-agent automated penetration testing system that integrates a large model and reinforcement learning, including a penetration testing startup module, an information collection module, a vulnerability mining module, a vulnerability solution generation module, a vulnerability verification module and a report generation module.

[0054] It should be noted that in this embodiment, each module involves a professional agent group, including: a Web application framework vulnerability agent group, a middleware vulnerability agent group, and a system transmission framework vulnerability agent group; the corresponding agent group contains each corresponding type of exclusive mining agent. The agent group of each module has its own responsibilities, and the design concept and processing tasks refer to different tasks in the penetration test process.

[0055] Specifically, the penetration test initiation module is used to receive an input valid URL that can access a website and initiate a hybrid multi-agent automated penetration test process combined with a large model.

[0056] For example: receive the input intranet IP, start the entire project, and enter the information collection module.

[0057] For details, see Figure 2 As shown, the information collection module adopts a URL-based multi-dimensional data collection strategy to systematically collect relevant information of the target system from multiple data sources.

[0058] The following is the specific implementation process of the information collection module.

[0059] (21) First, select a large-scale reasoning model with a global perspective to carry out subsequent information collection and reasoning tasks to ensure the consistency and accuracy of the reasoning process. Optional models include open source 100 billion-level models such as Zhipu Qingyan, GPT-4o, Grok, and Tongyi Qianwen.

[0060] For example, choose Zhipu Qingyan, GPT-4o, Grok, Tongyi Qianwen and other open source large models.

[0061] (22) Through the combination of tool scheduling agents and tool libraries and appropriate human intervention, the tool combination is optimized to ensure comprehensive collection of system information. The process uses local resources to perform information collection tasks and saves intermediate data for further analysis. For example, an open source network scanning tool is first used to perform preliminary detection of the target assets, and asset detection technology is used to identify network assets. Subsequently, the port asset information and system fingerprints are deeply analyzed through network mapping technology. The tool scheduling agent further uses directory enumeration tools and open source dictionary resources to detect page service information. Finally, through the integrated information collection tool, a comprehensive analysis of the existence of subdomains, side stations, WHOIS information, website architecture, registration number query, content delivery network (CDN), and Web application firewall (WAF) is performed. The tool scheduling agent can automatically generate scheduling scripts, schedule local resources to perform information collection tasks, and save key system information in text form.

[0062] (23) In the process of information collection, the information filtering agent plays a vital role. The information filtering agent is responsible for classifying the collected data and filtering out duplicate data to reduce the resource consumption of large models in the subsequent reasoning process. Specifically, although the advanced port identification technology used in the network asset identification process has high effectiveness, it may also generate redundant information. In order to avoid the accumulation of redundant data, the present invention designs an efficient information filtering agent. The information filtering agent can identify and remove duplicate network architectures, open ports and service information, operating system and application version information, service fingerprints, detailed configuration of Web applications, directory structures, subdomains and side station information, WHOIS and filing data, CDN configuration details, WAF deployment status, network traffic patterns, and key information in social engineering research. After screening, the information filtering agent retains unique and key information to form a new intermediate operation data tag text.

[0063] (24) Based on the formation of new intermediate operation data labeled text data, the information processing process was further designed and implemented. This process is managed by a special information collection and scheduling agent, which divides the text data into fine parts and intelligently schedules the corresponding processing agents according to the information characteristics. Each processing agent combines the pre-built knowledge base to collect preliminary relevant service version information for the assigned information fragments, aggregates the data to form new text data, and enters the vulnerability mining module. For example, when Spring-Boot service information is identified, the information collection and scheduling agent dispatches this part of information to the Web application framework agent for information collection, so as to further collect relevant information about the version framework, including version information, relevant dependency package information, possible configuration file utilization information, etc.

[0064] (25) Finally, the user evaluates the new text data results to determine whether the information is sufficient and valid, and decides whether to conduct vulnerability mining operations, whether to collect information again, or exit the system.

[0065] For details, see Figure 3 As shown, the vulnerability mining module, based on the new text data content formed by the information collection module, adopts a multi-dimensional strategy to realize vulnerability mining, and integrates an adaptive learning mechanism to improve detection efficiency. Each service agent is equipped with a dedicated vulnerability library to enhance its detection and response capabilities.

[0066] The specific implementation process is as follows:

[0067] (31) First, dispatch the locally deployed rule-based fuzzy vulnerability scanning tool and its supporting template library resources. The core operation is to use the tool's efficient rule matching capabilities to comprehensively identify vulnerabilities in the target environment. After the scan is completed, the system will integrate the scan results and generate a detailed data text record. It should be noted that the information in the text has not yet been confirmed in terms of validity and needs to be analyzed and verified through subsequent operations. For example, the scan may detect multiple serious, high-risk, medium-risk or low-risk risks such as CVE-2021-21234, spring-detect, options-method, etc.

[0068] (32) As in stage (21), the vulnerability mining module will again select a large-scale reasoning model with a global perspective to carry out subsequent reasoning tasks and ensure the consistency and accuracy of the reasoning process.

[0069] (33) In this link, the vulnerability mining scheduling agent plays a key role, responsible for fusing the data text record (data output of the rule matching vulnerability scan) with the information collection module to form the content of the new text data. The specific operation includes classifying and identifying the service information of the scan results and distributing it to the corresponding service agent. Each service agent then uses its own vulnerability knowledge base to conduct in-depth mining and detailed description of the assigned vulnerability information. All analysis results will be integrated into an intermediate data set. For example, if the detailed data text record generated in stage (31) is analyzed and the CVE-2021-21234 (vulnerability number, from the CVE website) vulnerability is found in the target system, the vulnerability mining scheduling agent associates it with the Spring Boot framework and passes it to the agent responsible for Spring framework vulnerability mining for further processing. The vulnerability mining agent first queries the knowledge base to obtain a detailed description of the CVE-2021-21234 vulnerability. Based on the information provided by the information collection module to form the new text data, the system identifies the services related to Spring Boot, extracts relevant data fragments for the service, and inputs them into the Spring vulnerability mining agent. The vulnerability mining agent then dispatches information from the knowledge base to decide whether there are historical vulnerabilities associated with the service and records this finding.

[0070] (34) Based on the intermediate data set formed in stage (33), the operation framework is further expanded to introduce a vulnerability mining tool scheduling agent. The core responsibility of the vulnerability mining tool scheduling agent is to make a detailed distinction between service information and generate a scheduling script for the local framework vulnerability scanning tool to guide local resources to implement vulnerability scanning. The vulnerability mining recovery agent then processes the scan results and integrates the analysis results into a new intermediate data set. If there is no vulnerability scanning tool for a specific service framework in the local resources, this step will be skipped accordingly. For example, after identifying the Spring service, the local tool related to the Spring framework vulnerability scan is scheduled to obtain the CVE-2021-21234 vulnerability information; when the MySQL service is identified, if there is no local scanning resource for the tool, this step will be skipped.

[0071] (35) In the final stage, the vulnerability mining result processing agent takes on the core responsibility of comprehensively processing the intermediate data set text generated in stage (33) and the possible new intermediate data set text generated in stage (34). The specific operations include performing labeling operations to remove duplicate information and marking the number of hits for duplicate vulnerability information. Subsequently, the vulnerability mining result processing agent generates a comprehensive vulnerability mining report based on the prompt project template, which will be used in the subsequent vulnerability solution generation module.

[0072] For details, see Figure 4As shown, the vulnerability solution generation module, based on the vulnerability mining report text content, further integrates the local vulnerability scanning results based on proof of concept (POC), and converts the vulnerability mining report into an executable exploitation solution.

[0073] The specific implementation process is as follows:

[0074] (41) First, the locally deployed POC-based fast vulnerability scanning tool (localized deployment of vulnerability scanning tools compiled based on open source and closed source) is scheduled to perform a quick and in-depth scan of the test environment. Possible vulnerabilities in the system are quickly identified and the corresponding intermediate data set is generated. If no vulnerabilities are found during the execution process, no intermediate data set is generated.

[0075] (42) Similar to stages (21) and (32), the vulnerability solution generation module will select a large-scale reasoning model with a global perspective to carry out subsequent reasoning tasks and ensure the consistency and accuracy of the reasoning process.

[0076] (43) In this link, the vulnerability exploitation scheme generation scheduling agent plays a key role, responsible for integrating the POC-based vulnerability mining data generated in stage (41) and the content of the vulnerability mining results generated in stage (35). The vulnerability exploitation scheme generation scheduling agent first classifies and identifies the service information of the scanning results and distributes the information to the corresponding service agents. Each service agent then combines its specialized vulnerability knowledge base to conduct in-depth mining of the assigned vulnerability information and forms a vulnerability verification scheme based on the known information. Finally, each service agent generates separate intermediate processing data based on the vulnerability name, which will enter the vulnerability verification module. For example, based on the information of the vulnerability mining results text generated in stage (35), if the CVE-2021-21234 vulnerability is detected in the Spring service, the vulnerability exploitation scheme generation scheduling agent will schedule the Spring vulnerability exploitation scheme generation agent to combine the CVE-2021-21234 reproduction method in the knowledge base to generate an exploit scheme for the vulnerability and generate an exploit text about CVE-2021-21234.

[0077] For details, see Figure 5 As shown, the vulnerability verification module is intended to further verify the operations generated by the vulnerability solution generation module to screen and confirm the accurate vulnerability information that exists.

[0078] The specific implementation process is as follows:

[0079] (51) Similar to stages (21), (32), and (42), this module will select a large-scale reasoning model with a global perspective to carry out subsequent reasoning tasks and ensure the consistency and accuracy of the reasoning process.

[0080] (52) Based on the multiple vulnerability exploitation scenario texts proposed in stage (43), vulnerability exploitation scripts were further developed. These scripts are designed to schedule agents corresponding to specific services to achieve automated vulnerability exploitation script generation. For example, based on the exploit scenario text for the CVE-2021-21234 vulnerability generated in Section 4.3, the Spring vulnerability verification agent will generate the corresponding exploitation script.

[0081] (53) Based on the vulnerability exploitation script generated in stage (52), the system schedules local environment resources for execution. During the execution process, the service type-specific agent monitors the results of vulnerability verification and conducts a comprehensive evaluation. The service type-specific agent will determine whether the execution is successful and analyze whether script exceptions occur during the execution process. These exceptions may affect the accuracy of the final result. Based on the evaluation results, the service type-specific agent will make corresponding decisions, including whether to regenerate the script or remove the vulnerability record. The successfully verified vulnerability information (including description and script information) will be recorded and summarized to form intermediate text data. For example, in the exploitation scheme text for CVE-2022-22947 and CVE-2021-21234 vulnerabilities generated in stage (43), if CVE-2021-21234 is successfully verified, the information will be recorded in the intermediate text of this stage; if CVE-2022-22947 fails to be verified, the service type-specific agent will not record the vulnerability information. For example, in stage (43), an exploit text for the CVE-2022-22947 vulnerability and an exploit text for the CVE-2021-21234 vulnerability are generated. If CVE-2021-21234 is successfully verified, the intermediate text entering this stage is recorded. If CVE-2022-22947 fails to be verified by the service type-specific intelligent agent, the vulnerability information is not recorded.

[0082] For details, see Figure 6 As shown, the report generation module is based on the intermediate text data generated in stage (53) and combined with a specific security assessment report template. Through the preset input prompt, the module can automatically generate a detailed security assessment report. In addition, this module is also responsible for recording the operation data generated in the entire security assessment process and storing it in the database to construct a sample set for agent training.

[0083] The specific process is as follows:

[0084] (61) Similar to stages (21), (32), (42), and (52), the report generation module will select a large-scale reasoning model with a global perspective to carry out the reasoning task of the current module and ensure the consistency and accuracy of the reasoning process.

[0085] (62) Using the selected model, this module will generate a detailed and readable security report that meets specific specifications, aiming to provide clear and understandable information to security analysts and decision makers so that they can take appropriate security measures.

[0086] (63) Throughout the entire process, the report generation module will record the intermediate data sets generated by all operations and store them in the database. Through these data, a complete set of agent training samples will be constructed, which will be used for feedback learning to continuously strengthen and optimize the performance and capabilities of all participating agents.

[0087] Furthermore, the present invention provides a multi-agent automated penetration testing system that integrates a large model and reinforcement learning. As a preferred method, in addition to steps S1 to S6, it also includes an update module for regularly updating the agent vulnerability library and tool library.

[0088] In order to ensure the continued effectiveness of the agent system, the present invention implements a regular update mechanism to supplement and optimize the vulnerability library information of each agent, as well as to expand the available tools in the tool library.

[0089] The following is a detailed description of the mechanism:

[0090] Vulnerability library information supplement: Regularly review and update the vulnerability library of each agent to include the latest vulnerability information, attack vectors, defense strategies and related reference data. This process ensures that the agent maintains the latest knowledge base in vulnerability identification and exploitation, thereby improving the overall detection capability and response speed of the system.

[0091] Tool library expansion: Regularly identify and evaluate emerging security testing tools and include them in the tool library. The selected tools must undergo a rigorous screening process to ensure that they can enhance the functionality of existing agents and improve the efficiency and accuracy of vulnerability discovery and verification.

[0092] See also Figure 7 The present embodiment also discloses a multi-agent automated penetration testing method integrating a large model and reinforcement learning, including:

[0093] S701, penetration test initiation step: receiving a valid URL of an accessible website input, and initiating a hybrid multi-agent penetration test based on a large model;

[0094] S702, information collection step: collecting detailed information of classified websites through multi-agent collaboration;

[0095] S703, vulnerability mining step: identify vulnerabilities through multi-agent collaboration and gradually improve the effectiveness of decision-making through deep learning feedback mechanism;

[0096] S704, vulnerability solution generation step: customizing relevant vulnerability exploitation strategies through multi-agent collaboration;

[0097] S705, vulnerability verification step: verify the effectiveness of the vulnerability through multi-agent collaboration, collect the results and use the deep learning feedback mechanism to gradually improve the decision accuracy;

[0098] S706, report generation step: multiple agents generate a safety report based on the results of the previous steps and feed it back to relevant agents to enhance deep learning performance.

[0099] The specific implementation of a multi-agent automated penetration testing method that integrates a large model and reinforcement learning is the same as the multi-agent automated penetration testing system that integrates a large model and reinforcement learning, and this embodiment will not be repeated.

[0100] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-agent automated penetration testing system integrating large models and reinforcement learning, characterized in that: include: A penetration test initiation module, which is used to receive a valid URL of an accessible website as input and initiate a hybrid multi-agent automated penetration test process combined with a large model; The information collection module is used to generate scheduling scripts through tool scheduling agents, schedule localized tool libraries, collect website information, and process information through application layer, network layer and transport layer through the set agents, identify relevant service information, filter out duplicate data, and extract information at each level; The vulnerability mining module is used to integrate the information collected by the information collection module at the application layer, network layer and transport layer, and combines it with the localized vulnerability scanning tool based on YAML template rules. It uses the Web application framework vulnerability mining agent group, the middleware vulnerability mining agent group and the system transport layer service vulnerability mining agent group to perform vulnerability scanning and identification. The vulnerability mining text processing scheduling agent is responsible for coordinating the activities of different service agents to form a vulnerability mining report. The vulnerability solution generation module is used to integrate the vulnerability information identified by the vulnerability mining module and the localized deployment scanning results based on the vulnerability scanning tool. The vulnerability mining information processing scheduling agent schedules the corresponding professional vulnerability solution generation agent group, and customizes the corresponding vulnerability exploitable solution for each specific vulnerability in combination with the vulnerability library information. The vulnerability verification module is used to operate on multiple different vulnerability information based on the generated vulnerability exploitation schemes. The vulnerability verification scheduling agent is responsible for coordinating the professional vulnerability verification agent group including the Web application framework, middleware and transport layer to achieve concurrent operation. Each vulnerability information generates an executable vulnerability exploitation script, dispatches local resources to execute the script and analyze the results, and generates vulnerability information and exploitation scripts. The report generation module is used to integrate the results generated by the information collection module, vulnerability mining module, vulnerability solution generation module and vulnerability verification module, provide a variety of model options to optimize data processing, perform filtering, exclude junk and duplicate information functions, generate a security assessment report based on a predetermined security report template, and convert the data processed by the information collection module, vulnerability mining module, vulnerability solution generation module and vulnerability verification module into training samples to enhance the performance of each intelligent agent and realize a feedback learning mechanism.

2. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 1 is characterized in that: The information collection module comprises: A large model selection unit is used to select and process a large model to carry out information collection and reasoning tasks; The collection decision acquisition unit is composed of a tool scheduling agent that designs the information collection process based on the predetermined workflow and tool library information, decides which tool combination to use to achieve more comprehensive information collection, and obtains the collection decision; An information collection text generation unit, used for using the collection decision to schedule local tool resources to capture information and form an information collection text; A data deduplication unit, used for filtering out duplicate data using an information filtering agent; An intermediate data set text generation unit is used to use an information collection scheduling agent to schedule different agents to divide the collected data, and the processing agents of the application layer, the network layer and the transport layer process the collected information to form an intermediate data set text; The user decision unit is used to provide the generated intermediate data set text to the user for reading and decision-making, to decide whether to perform vulnerability mining operations, whether to re-collect information or whether to exit the system.

3. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 1 is characterized in that: The vulnerability mining module includes: The information processing and scheduling unit is used to classify the collected results based on the intermediate data set text collected by the information collection module and the results generated by the local vulnerability scanning tool according to the YAML format matching rules. The information collection document processing agent classifies the collected results and distributes them to the corresponding dedicated agents for analysis; A scanning task execution unit is used to use the information collection document processing scheduling agent to generate and schedule the corresponding framework vulnerability scanning tool, execute the scanning task, and collect the scanning results; The agent grouping and vulnerability library configuration unit is used to divide the scheduling information collection document processing scheduling agent into a Web application framework vulnerability mining agent group, a Web middleware vulnerability mining agent group, and a system transport layer service vulnerability mining agent group, and correspond to the corresponding vulnerability library information to build an information knowledge base; The service information-driven vulnerability mining unit is used to use the vulnerability mining text processing scheduling agent to coordinate the activities of the Web application framework vulnerability mining agent group, the middleware vulnerability mining agent group, and the system transport layer service vulnerability mining agent group, and then coordinate the service type-specific agents corresponding to the degree to perform vulnerability mining tasks for targeted services and form a vulnerability mining report.

4. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 1 is characterized in that: The vulnerability solution generation module includes: The vulnerability data processing and scheduling unit is used to schedule the vulnerability scanning results of the localization tool based on the report generated by the vulnerability mining module and the vulnerability mining information processing agent based on the POC, classify the result text and assign it to the corresponding service type dedicated agent for processing; The data reasoning model selection unit is used to select a large model suitable for vulnerability data reasoning to support the subsequent agent reasoning process; The agent grouping and solution generation unit is used to divide the vulnerability exploitation solution generation module into an agent group based on Web application vulnerability exploitation solution generation, an agent group based on middleware vulnerability exploitation solution generation, and an agent group based on system transmission vulnerability exploitation solution generation according to different vulnerability types. Each agent group performs corresponding reasoning tasks and generates vulnerability exploitation solutions.

5. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 1 is characterized in that: The vulnerability verification module includes: A vulnerability exploitation scheme scheduling unit, used to use the vulnerability exploitation scheme information to schedule the agent allocation and schedule the corresponding service type-specific agent, and select a large model to execute the reasoning process of the vulnerability exploitation script; The agent grouping and task allocation unit is used to divide the vulnerability verification module into a Web application service framework vulnerability exploitation script generation agent group, a middleware vulnerability exploitation script generation agent group, and a system transport layer vulnerability exploitation script generation agent group, which are responsible for the corresponding vulnerability mining tasks; The vulnerability exploitation script generation and testing unit is used to schedule the corresponding service type-specific intelligent agent to generate vulnerability exploitation scripts and schedule local resources for testing; if the test is successful, vulnerability verification data is generated; otherwise, it is determined whether it is a coding problem. If it is a coding problem, an error log training sample is generated and the vulnerability exploitation solution scheduling unit is repeatedly executed. If it is not a coding problem, the vulnerability information is not recorded.

6. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 1 is characterized in that: The report generation module comprises: A vulnerability verification data processing unit, used for selecting a suitable large model to process the vulnerability verification data generated by the vulnerability verification module; A report compilation unit for generating a readable and safe report using the selected large model; The data recording and sample construction unit is used to record the data generated by all operations in the process into the database to form a complete agent training sample set.

7. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 1 is characterized in that: It also includes an update module for regularly updating the agent vulnerability library and tool library.

8. The multi-agent automated penetration testing system integrating large models and reinforcement learning according to claim 7 is characterized in that: The update module comprises: A vulnerability library information supplement unit is used to regularly review and update the vulnerability library information of the intelligent agent; The tool library expansion unit is used to regularly identify and evaluate new security testing tools and incorporate them into the tool library.

Images