Automatic penetration testing method and device based on large language model multi-agent cooperation

Abstract

The application belongs to the technical field of network security and large language model, and discloses an automatic penetration testing method and device based on large language model multi-agent cooperation, which comprises the following steps: a large language model is used to analyze a penetration testing target or task description input by a user into a basic task chain; a basic task is taken in sequence as a current task, and a corresponding preliminary basic operation is generated by using the large language model; an external experience knowledge base and a negative experience base are constructed, the preliminary basic operation is optimized based on a retrieval enhancement generation technology, and the optimized basic operation is executed; a penetration report is generated based on an execution result, and a negative experience is generated and stored in the negative experience base when the execution result is a failure; based on the basic task chain and the penetration report, the large language model is used to adjust the basic task chain, and automatic penetration testing operation is continued based on the adjusted basic task chain. The application effectively improves the efficiency and accuracy of penetration testing and enhances the network security defense capability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network security and large language model, and particularly relates to an automatic penetration testing method and device based on large language model multi-agent cooperation. BACKGROUND

[0002] Penetration testing is an important means of actively assessing network security. The traditional method mainly relies on manual operation and semi-automatic tools. Although its process can be customized for specific system detection, it still has significant drawbacks, such as requiring a large amount of manpower and low efficiency, and the depth and comprehensiveness of the test largely depend on the experience level of experts, lacking universality and efficiency.

[0003] At present, large language models of artificial intelligence have made significant progress, and agent technology based on large language models has also developed rapidly, gradually extending from basic text generation to automated decision-making, complex system optimization and other advanced tasks. However, in the application of penetration testing, there are still many challenges. On the one hand, task execution is easily affected by initial bias, and bias accumulation can make the test path deviate from the expected path and interfere with the test process. On the other hand, large language models often have "hallucination" problems, generating seemingly reasonable but unfeasible content, which is due to the lack of vertical domain knowledge, which cannot guarantee the effectiveness of the penetration testing task.

[0004] Therefore, there is an urgent need for new solutions to solve these problems, improve the efficiency and accuracy of penetration testing, and enhance the network security defense capability to cope with the increasingly severe network security situation. SUMMARY

[0005] The purpose of the present application is to provide an automatic penetration testing method and device based on large language model multi-agent cooperation, which effectively improves the efficiency and accuracy of penetration testing and enhances the network security defense capability.

[0006] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0007] First aspect: provide an automatic penetration testing method based on large language model multi-agent cooperation, comprising:

[0008] Using a large language model to parse the penetration testing target or task description input by the user into a basic task chain, the basic task chain containing a plurality of basic tasks arranged in order;

[0009] Take one basic task in the basic task chain as the current task in order, and use a large language model to generate a preliminary basic operation corresponding to the current task;

[0010] constructing an external experience knowledge base and a negative experience base, and optimizing the preliminary basic operation based on a retrieval enhancement generation technology and performing the optimized basic operation;

[0011] generating a penetration brief based on the execution result of the optimized basic operation, and storing negative experience into the negative experience base when the execution result is failure;

[0012] adjusting the basic tasks in the basic task chain that have not been executed by using a large language model according to the basic task chain and the penetration brief, and continuing the automatic penetration testing operation based on the adjusted basic task chain.

[0013] The following also provides several optional modes, but not as an additional limitation to the above overall scheme, just a further supplement or preferred, without technical or logical contradiction, each optional mode can be combined alone for the above overall scheme, but also can be combined between multiple optional modes.

[0014] As a preferred, the external experience knowledge base is a multi-layer experience base, and each layer of the experience base stores external experience knowledge of a penetration testing stage.

[0015] As a preferred, the construction process of the external experience knowledge base is as follows:

[0016] obtaining a penetration testing internal report and converting it into vector data, and storing the vector data into a vector database;

[0017] obtaining network sharing information, removing HTML tags in the network sharing information and extracting text information, dividing the text information into text blocks, converting the text blocks into vector data, and storing the vector data into a vector database.

[0018] As a preferred, the optimization of the preliminary basic operation based on the retrieval enhancement generation technology includes:

[0019] filtering the preliminary basic operation by using the negative experience base, if the preliminary basic operation is filtered, feeding back the preliminary basic operation and the negative experience that filters the preliminary basic operation to a large language model used to generate the preliminary basic operation, and returning to regenerate the preliminary basic operation by using the large language model;

[0020] otherwise, according to the penetration testing stage where the preliminary basic operation is located, retrieving the most relevant external experience from the external experience knowledge base, generating the optimized basic operation by using the large language model based on the preliminary basic operation and the most relevant external experience.

[0021] As a preferred, the filtering of the preliminary basic operation by using the negative experience base includes:

[0022] retrieving multiple negative experiences most similar to the preliminary basic operation in the negative experience library based on cosine similarity;

[0023] inputting the retrieved multiple negative experiences and the preliminary basic operation into the large language model combined with the prompt word template, outputting a conclusion of whether the preliminary basic operation is filtered by the large language model, and outputting the negative experience of filtering the preliminary basic operation when the conclusion is filtered.

[0024] As a preferred, when performing the optimized basic operation, the tool dynamic call based on the MCP architecture is implemented, first, the penetration testing tool is packaged as a tool agent with a structured description; second, when the optimized basic operation is obtained, the basic operation and the structured description of the tool agent are spliced into a prompt word template, the most suitable penetration testing tool is selected using the large language model, the key parameters in the optimized basic operation are injected into the parameter template of the selected penetration testing tool, the security Shell command is generated and executed.

[0025] As a preferred, the content of the penetration report includes the optimized basic operation, the response of the penetration testing tool performing the optimized basic operation, the environment state of the measured object and the success flag;

[0026] The negative experience is in the form of a triple, which contains the optimized basic operation, the response of the penetration testing tool performing the optimized basic operation, and the environment state of the measured object.

[0027] As a preferred, when adjusting the basic task chain, the adjustment rule followed by the large language model is:

[0028] If the success flag in the penetration report is failure, or the environment state of the measured object deviates from the expectation by more than a threshold value, a new basic task chain is generated; otherwise, the original basic task chain is maintained.

[0029] Second aspect: provide an automatic penetration testing device based on large language model multi-agent cooperation, comprising:

[0030] The task generation module is configured to parse the penetration testing target or task description input by the user into a basic task chain using a large language model, wherein the basic task chain contains a plurality of basic tasks arranged in order.

[0031] The task execution module is configured to take one basic task in the basic task chain as a current task in order, generate a preliminary basic operation corresponding to the current task using a large language model, and execute the optimized basic operation.

[0032] The multi-level experience fusion module is configured to construct an external experience knowledge base and a negative experience library, and optimize the preliminary basic operation based on retrieval enhancement generation technology to generate an optimized basic operation.

[0033] a penetration brief generation module configured to generate a penetration brief based on the execution result of the optimized basic operation, and store negative experience in a negative experience library when the execution result is a failure;

[0034] a task adjustment module configured to adjust unexecuted basic tasks in the basic task chain using a large language model according to the basic task chain and the penetration brief, and continue the automatic penetration testing operation based on the adjusted basic task chain.

[0035] The automatic penetration testing method and device based on large language model multi-agent collaboration provided by the present application have the following advantages compared with the prior art:

[0036] (1) The understanding and generation capabilities of the large language model for task logic, combined with multi-agent collaboration and external experience knowledge base, realize the automation of penetration testing, effectively reduce the dependence on human intervention, and improve the efficiency of penetration testing. (2) The basic task chain can be dynamically modified according to the environmental state feedback to alleviate the task deviation phenomenon and ensure the adaptability and effectiveness of the task in complex environments. (3) The failure experience recording mechanism provided by the negative experience library reduces the incidence of repetitive error operations by automatically capturing and reusing negative experience, and improves the reliability of task execution. (4) With the help of external experience knowledge and retrieval augmented generation (RAG) technology to optimize operations, compared with traditional full library retrieval, it has higher retrieval performance under large-scale data, reduces the "hallucination" problem of large language models, and improves the success rate and accuracy of task execution. (5) The MCP architecture is used to realize the dynamic collaboration and calling of tools by multi-agent, which breaks through the limitation of traditional tool chains, ensures safety isolation, and improves execution efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flowchart of the automatic penetration testing method based on large language model multi-agent collaboration of the present application;

[0038] Figure 2 is a framework diagram of the automatic penetration testing device based on large language model multi-agent collaboration of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0041] Embodiment 1:

[0042] As shown in the embodiment, an automatic penetration testing method based on large language model multi-agent cooperation is provided, which specifically includes the following steps: Figure 1

[0043] Step 1, parse the user input penetration testing target or task description into a basic task chain using a large language model, and the basic task chain contains a plurality of basic tasks arranged in order.

[0044] Receive user input penetration testing target or task description, generate basic task chain through zero sample thought chain (CoT), and execute basic tasks in basic task chain in order and record remaining unfinished basic tasks for subsequent feedback of environment state after execution and trigger task adjustment.

[0045] When generating the basic task chain, the large language model is guided to generate by means of the prompt word template. The embodiment provides a prompt word template example as: “You are a professional network security researcher, and you are currently helping a company detect potential security risks in its system. In order to complete your task, please create a step-by-step task list based on the following task description, ensuring that each task is a basic, actionable work unit. Please think step by step. Task goal: {penetration testing target or task description}; basic task chain output in the following format: 1. Task 1; 2. Task 2; …; n. Task n.” In this way, the natural language processing capability of the large language model is utilized, combined with external retrieval information, to generate a task sequence with clear logic and strong operability.

[0046] Step 2, take one basic task in the basic task chain as the current task in order, and generate the preliminary basic operation corresponding to the current task using the large language model.

[0047] ​The current task is combined with the designed prompt word template as input, and a large language model is used to generate a preliminary basic operation (basic operation containing an operation instruction) that can specify the basic task. The prompt word template designed in this embodiment is as follows: "You are a professional network security researcher, and you are currently performing a {stage type} stage (stage type: probe / vulnerability assessment / exploitation) of penetration testing. According to the current task {Task} (complete task chain context: {Task Chain}) in the basic task chain, generate an operation instruction that can be parsed by the MCP architecture. Requirements: 1) Clearly specify the penetration testing tool name (limited to registered tools such as Nmap / Metasploit / Sqlmap); 2) Include complete parameters (parameter format must conform to the JSON format specification of the Tool Agent); 3) Output in the standard operation instruction format: 'Operation instruction: [tool name] [parameter key-value pair]'. Note: Do not generate non-registered tool instructions, and the parameter values must be dynamically injectable."

[0048] Since the basic operation generated directly by the large language model may have errors, this embodiment introduces a multi-level experience fusion mechanism that interacts with an external experience knowledge base to determine the best basic operation. This embodiment further optimizes the operation in combination with the multi-level experience fusion mechanism to ensure effective execution in the target environment.

[0049] Step 3, build an external experience knowledge base and a negative experience base, and optimize the preliminary basic operation based on the retrieval enhancement generation technology, and execute the optimized basic operation.

[0050] The external experience knowledge base of the present application is a multi-layer experience base, and each layer of the experience base corresponds to the storage of external experience knowledge of a penetration testing phase. For example, the present embodiment constructs three layers of experience bases according to the penetration testing phases, namely the "probe" phase, the "vulnerability assessment" phase and the "exploitation" phase. When constructing the external experience knowledge base, the internal report of the penetration test is obtained and converted into vector data, and the vector data is stored in the vector database; at the same time, network sharing information (such as experience sharing crawled from security blogs) is obtained, in order to ensure that the data has suitable structural characteristics for subsequent vectorization processing, for the network sharing information, the HTML tags in the network sharing information are removed and the text information is extracted (such as using the BeautifulSoup tool), the text information is segmented into text blocks (such as using the text segmentation tool provided by LangChain), the text blocks are converted into vector data (such as using the OpenAI Embeddings model), and the vector data is stored in the lightweight Chroma vector database. When storing, according to the penetration testing phase where the original data is located, the vector data is stored in the corresponding layer of the experience base, and the external experience knowledge record JSON format example: { "stage label": "probe", "operation instruction": "nmap -sS -p 1-1000192.168.1.1", "tool response": "discovered open ports 22, 80, 443", "environment state": "target host alive, SSH and Web service running", "success flag": true}. In addition, the negative experience base constructed in this step is initialized as an independent negative experience base, and the contents in the negative experience base are added according to the basic operation execution results.

[0051] When the retrieval enhancement generation technology is executed, the preliminary basic operation is filtered by using the negative experience base. If the preliminary basic operation is filtered, the preliminary basic operation and the negative experience of filtering the preliminary basic operation are fed back to the large language model used to generate the preliminary basic operation, so that the large language model re-generates the preliminary basic operation, that is, returns to step 2 to re-execute; otherwise, according to the penetration testing phase where the preliminary basic operation is located, the most relevant multiple external experiences in the external experience knowledge base are retrieved; based on the preliminary basic operation and the most relevant multiple external experiences, the large language model is used to generate the optimized basic operation.

[0052] When performing the filtering operation, the negative experience similar to the preliminary basic operation in the negative experience base is retrieved based on the cosine similarity, and the large language model strictly judges the failure mode and similarity risk thereof. If it passes, the preliminary basic operation is not filtered; otherwise, the preliminary basic operation is filtered.

[0053] The embodiment provides a prompt word template example of negative experience library filtering: "You are a professional network security researcher, and are currently performing a security assessment of a penetration test operation. Please make a negative experience filtering judgment according to the following input: input information: initial basic operation: {Initial_Operation}; current penetration stage: {Current_Stage}; negative experience library search record: {Negative_Experience_Database}, judgment requirements: 1) strictly compare the initial basic operation with similar failure modes searched out from the negative experience library; 2) identify the similarity risk of operation instructions, tool parameters and execution environment, and output format: if filtering is needed: {"filtering judgment": true, "matching negative experience": ["operation instruction", "tool response", "environment state"], "risk reason": "specific risk description"}; if filtering is not needed: {"filtering judgment": false}".

[0054] The embodiment provides a prompt word template example of regenerating an initial basic operation: "You are a professional network security researcher, and are currently performing a {Current_Stage} stage of a penetration test. Your initial operation is identified as a potential risk operation by a security filtering system, and a safe and reliable alternative operation needs to be regenerated. Background information: original basic task: {Original_Task}; filtered initial operation: {Filtered_Operation}; filtering reason and negative experience: {Negative_Experience_Detail}; complete task chain context: {Task_Chain}, regeneration requirements: 1) keep the core goal of the original basic task unchanged; 2) strictly avoid failure modes (tool selection, parameter configuration, execution time) in the negative experience; 3) select a safer and more reliable tool and parameter combination; 4) ensure that the new operation instruction meets the MCP architecture analysis standard, output format: 'operation instruction: [alternative tool name] [optimized parameter key-value pair]', note: the newly generated operation must be functionally equivalent to the original task requirement, but completely avoids the known failure path in the implementation way".

[0055] For the filtered initial basic operation, the initial basic operation is first vectorized into a vector form in a high-dimensional space as an input vector. Then, cosine similarity between the input vector and vector data in an experience library corresponding to a penetration test stage to which the input vector belongs is calculated, so as to quantify the similarity degree of the vectors in a semantic space. For vectors and , is the total number of vectors, is the i-th vector , an element, is a vector The formula for calculating the cosine similarity of the first element in the vector is shown in equation (1).

[0056] (1)

[0057] The present embodiment realizes logical selection hierarchical retrieval through label filtering, adopts a retrieval algorithm based on cosine similarity to calculate the similarity of the external experience in the experience library, recalls the top_3 external experience most relevant to the preliminary basic operation from the experience library, and analyzes the recalled top_3 experience and the preliminary basic operation together as a prompt word by the large language model to select the optimal operation step, that is, to obtain the optimized basic operation. The prompt word template for optimizing the preliminary basic operation provided in the present embodiment is: “You are a professional network security researcher, currently in the {Current stage} stage (stage type: detection / vulnerability assessment / exploitation). Please generate an optimal operation instruction containing specific tool names and complete parameters according to the preliminary basic operation {Execute Command} and the top_3 external experience {RAG} retrieved, and output it in the format of [stage label] operation instruction: {optimized instruction}.” In this way, the historical experience is used to assist the decision of the large language model, thereby improving the accuracy and reliability of the operation step.

[0058] When executing the optimized basic operation, the MCP architecture is used to realize dynamic tool calling. First, the penetration testing tools (such as Nmap / Metasploit / Sqlmap tools, etc.) are encapsulated into tool agents with structured descriptions, and the function description is stored in a standardized JSON format, such as {"tool":"nmap","args":{"target":"192.168.1.1"}}. Second, when the optimized basic operation is obtained, the basic operation and the structured description of the available penetration testing tools are spliced into a prompt word template, the large language model is used to select the most suitable penetration testing tool, the key parameters (such as target IP and scan type) in the basic operation instruction are parsed and injected into the parameter template of the selected penetration testing tool, a security Shell command is generated and executed; finally, the original output (execution result) of the penetration testing tool is captured and spliced with the optimized basic operation as feedback information.

[0059] Step 4, generate a penetration report based on the execution result of the optimized basic operation, and generate negative experience and store it in the negative experience library when the execution result is failure.

[0060] After each round of operation step execution, a penetration report is generated based on the basic operation and tool response, and it is determined whether the operation is successful. The report content includes: optimized basic operation, response of penetration testing tool executing the optimized basic operation, environment state of the measured object and success flag. In order to effectively inhibit the repetition and invalid operation of the model in the penetration testing process, a failure experience recording mechanism is introduced. If the operation instruction execution fails, the report content is packaged as a "negative experience triple" as negative experience and stored in an independent negative experience library. In subsequent task execution, the model will use these negative samples to avoid such invalid or failed operations.

[0061] The prompt word template of the penetration report generation part in this embodiment is: "You are a professional network security researcher, and you are currently performing penetration testing operations. Please generate strictly standardized output according to the following input:

[0062] Part 1. For the execution instruction {Execute Command} and the tool original response {ExecutionResults}, generate a JSON format report: {"operation instruction":"...", "tool response":"...", "environment state":"...", "success flag":true / false};

[0063] Part 2. If the success flag=false, an additional negative experience triple is generated, format: {"negative experience":["operation instruction", "tool response", "environment state"]}.

[0064] If the success flag=true, only the JSON format penetration report of Part 1 is output, and the output requirement is: prohibit any explanatory words, and only return a parsable JSON object.

[0065] Step 5, according to the basic task chain and the penetration report, adjust the unexecuted basic tasks in the basic task chain using a large language model, and continue the automatic penetration testing operation based on the adjusted basic task chain.

[0066] Environment state evaluation: after each round of operation step execution, the penetration report is given to the large language model as a prompt word to evaluate in combination with the current basic task chain. If it is judged that the basic task chain needs to be adjusted, a new basic task chain is generated according to the evaluation. The new basic task chain retains the executed tasks, and the unexecuted tasks are adjusted according to the evaluation; if the basic task chain does not need to be modified, the applicability of the existing basic task chain is directly confirmed, and the task execution process is allowed to continue according to the established plan.

[0067] The embodiment provides a task chain adjustment prompt word template example: "You are a professional network security researcher, and are currently performing a penetration test task. Please strictly adjust the decision according to the environment state change record ({environment state}) and the success flag ({success flag}) in the penetration report, and combine the original basic task chain {Task Chain} to make dynamic adjustment decisions: if the report shows that the operation fails (success flag = false) or the environment state deviates from the expected value by more than a threshold (reference port closed / service version changed / IP unreachable), a new task chain is generated (the executed basic tasks {Task1~k} are retained, and the unexecuted basic tasks are replanned); otherwise, the original task chain is maintained. The output format must be: 1. Task 1 ; 2. Task 2 ; … ; n.Task n (strictly consistent with the original format)". Through task chain adjustment, the task chain is ensured to adapt to the changing test environment, task deviation is avoided, and the success rate of penetration testing is improved.

[0068] Embodiment 2

[0069] As shown in Figure 2 The embodiment provides an automatic penetration testing device based on a large language model and multi-agent cooperation, comprising:

[0070] A task generation module is configured to parse a penetration testing target or task description input by a user into a basic task chain using a large language model, wherein the basic task chain comprises a plurality of basic tasks arranged in order.

[0071] A task execution module is configured to take one basic task in the basic task chain as a current task in order, and generate a preliminary basic operation corresponding to the current task using a large language model; and is further configured to execute the optimized basic operation.

[0072] A multi-level experience fusion module is configured to construct an external experience knowledge base and a negative experience base, and optimize the preliminary basic operation based on a retrieval enhancement generation technology to generate an optimized basic operation.

[0073] A penetration report generation module is configured to generate a penetration report based on the execution result of the optimized basic operation, and store negative experience in the negative experience base when the execution result is a failure.

[0074] A task adjustment module is configured to adjust the unexecuted basic tasks in the basic task chain using a large language model according to the basic task chain and the penetration report, and continue the automatic penetration testing operation based on the adjusted basic task chain.

[0075] The specific limitations of the automatic penetration testing device based on a large language model and multi-agent cooperation can be referred to the limitations of the automatic penetration testing method based on a large language model and multi-agent cooperation in the foregoing, and will not be repeated here.

[0076] Any combination of the technical features in the above-described embodiments can be made, and for the sake of brevity, not all possible combinations are described, however, as long as there is no conflict, any combination of the technical features should be considered within the scope of the present disclosure.

[0077] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the application. It should be pointed out that, for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An automatic penetration testing method based on large language model multi-agent cooperation, characterized in that, The automatic penetration testing method based on the large language model multi-agent cooperation comprises the following steps: The large language model is used to parse the penetration testing target or task description input by the user into a basic task chain, wherein the basic task chain comprises a plurality of basic tasks arranged in order; One basic task in the basic task chain is taken as a current task in sequence, and the large language model is used to generate a preliminary basic operation corresponding to the current task; An external experience knowledge base and a negative experience base are constructed, and the preliminary basic operation is optimized based on a retrieval enhancement generation technology, and the optimized basic operation is executed; A penetration brief is generated based on the execution result of the optimized basic operation, and a negative experience is generated and stored in the negative experience base when the execution result is a failure; According to the basic task chain and the penetration brief, the large language model is used to adjust the unexecuted basic tasks in the basic task chain, and the automatic penetration testing operation is continued based on the adjusted basic task chain.

2. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 1, characterized in that, The external experience knowledge base is a multi-layer experience base, and each layer of the experience base stores external experience knowledge of one penetration testing stage.

3. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 1, characterized in that, The construction process of the external experience knowledge base is as follows: Vector data is obtained by converting a penetration testing internal report, and the vector data is stored in a vector database; Network sharing information is obtained, HTML tags in the network sharing information are removed, and text information is extracted, the text information is segmented into text blocks, the text blocks are converted into vector data, and the vector data is stored in the vector database.

4. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 1, characterized in that, The preliminary basic operation is optimized based on the retrieval enhancement generation technology, comprising the following steps: The preliminary basic operation is filtered by using the negative experience base, if the preliminary basic operation is filtered, the preliminary basic operation and the negative experience for filtering the preliminary basic operation are fed back to the large language model used for generating the preliminary basic operation, and the preliminary basic operation is regenerated by using the large language model; Otherwise, according to the penetration testing stage where the preliminary basic operation is located, a plurality of most relevant external experiences are retrieved from the external experience knowledge base, and the optimized basic operation is generated by using the large language model based on the preliminary basic operation and the plurality of most relevant external experiences.

5. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 4, characterized in that, The preliminary basic operation is filtered by using the negative experience base, comprising the following steps: A plurality of negative experiences most similar to the preliminary basic operation are retrieved from the negative experience base based on cosine similarity; The plurality of negative experiences and the preliminary basic operation retrieved are input into the large language model in combination with a prompt word template, a conclusion of whether the preliminary basic operation is filtered is output by the large language model, and the negative experience for filtering the preliminary basic operation is output together when the conclusion is filtered.

6. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 1, characterized in that, When the optimized basic operation is executed, the tool dynamic calling is realized based on the MCP architecture, first, the penetration testing tool is packaged into a tool agent with a structured description; second, when the optimized basic operation is obtained, the basic operation and the structured description of the tool agent are spliced into a prompt word template, the most suitable penetration testing tool is selected by using the large language model, the key parameters in the optimized basic operation are parsed into the parameter template of the selected penetration testing tool, a security Shell command is generated and executed.

7. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 1, characterized in that, The content of the penetration brief includes the optimized basic operation, the response of the penetration testing tool executing the optimized basic operation, the environment state of the measured object, and the success flag; The negative experience is in the form of a triple, which contains the optimized basic operation, the response of the penetration testing tool executing the optimized basic operation, and the environment state of the measured object.

8. The automatic penetration testing method based on large language model multi-agent collaboration according to claim 7, characterized in that, When adjusting the basic task chain, the adjustment rule followed by the large language model is: If the success flag in the penetration brief is failure, or the environment state of the measured object deviates from the expectation by more than a threshold value, a new basic task chain is generated; otherwise, the original basic task chain is maintained.

9. An automatic penetration testing device based on large language model multi-agent cooperation, characterized in that, The automatic penetration testing device based on the multi-agent cooperation of a large language model comprises: A task generation module is configured to parse the penetration testing target or task description input by a user into a basic task chain using a large language model, wherein the basic task chain contains multiple basic tasks arranged in order; A task execution module is configured to take one basic task in the basic task chain as a current task in sequence, and generate a preliminary basic operation corresponding to the current task using a large language model; and also configured to execute the optimized basic operation; A multi-level experience fusion module is configured to construct an external experience knowledge base and a negative experience library, and optimize the preliminary basic operation based on a retrieval enhancement generation technology to generate an optimized basic operation; A penetration brief generation module is configured to generate a penetration brief based on the execution result of the optimized basic operation, and generate a negative experience and store it in the negative experience library when the execution result is failure; A task adjustment module is configured to adjust the unexecuted basic tasks in the basic task chain using a large language model according to the basic task chain and the penetration brief, and continue the automatic penetration testing operation based on the adjusted basic task chain.

Images