Penetration test automation method and device based on large language model and ATTCK framework

Abstract

The invention discloses a method based on a large language model and ATTamp; the invention discloses a CK framework penetration test automation method and device, and the method comprises the steps: firstly carrying out the structural analysis of multi-source input information and tool output, and guaranteeing that key fields are not discarded; then combining a retrieval enhancement generation technology and a network security knowledge base to provide domain knowledge support for the large language model, so as to generate a model with ATTamp; a penetration test task tree marked by CK tactics, technologies and sub-technologies; on the basis, an optimal tool is automatically selected through a tool resource library and a multi-dimensional screening mechanism, an execution instruction is generated, and finally an execution result is returned to the input analysis module to form a self-adaptive optimization test closed loop. According to the method, semantic fidelity compression and standardization processing of long information can be realized aiming at the problems of large output format difference, more information redundancy and the like of different penetration testing tools, and efficient, explainable and auditory technical support can be provided for automatic penetration testing in a complex network environment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network security, and particularly relates to a penetration testing automation method and device based on a large language model and an ATT&CK framework. BACKGROUND

[0002] In recent years, with the continuous evolution of network attack technology and the continuous expansion of attack scale, penetration testing, as an important means to discover security vulnerabilities of information systems and verify the effectiveness of defense measures, plays a key role in the network security protection system. Traditional penetration testing relies on manual operation, simulating the attack process through information collection, vulnerability detection, exploit verification, privilege escalation, etc. Although it has advantages in depth and flexibility, it has problems such as long testing period, high cost, limited coverage, etc., and it is difficult to meet the current frequent and diversified security evaluation needs under the network environment. In order to improve efficiency and consistency, the industry and academia have begun to explore automated penetration testing methods, which automatically execute part of the penetration process through script-based task scheduling and integrated tool calling. However, this kind of method often calls tools in a fixed order, lacks overall control of the testing process, and cannot dynamically adjust the testing path according to the real-time intelligence obtained, resulting in redundant operations or missing key steps. At the same time, the output data formats of different penetration tools (such as Nmap, SQLMap, Metasploit, etc.) are significantly different, the information is redundant and lacks a unified structured processing mechanism, and key information is easily missed in the compression and transmission process, affecting the accuracy of subsequent decision-making.

[0003] The development of large language models (LLMs) has brought new opportunities for automated penetration testing. Some studies propose an automated penetration testing framework combined with LLMs, which drives task planning and tool invocation through natural language reasoning capabilities, uses penetration testing trees (PTTs) to record executed and pending steps, and dynamically adjusts testing strategies in cyclic iterations. This method improves the rigidity of the traditional process to some extent and can generate the next operation suggestion based on tool output. However, the existing LLM-based solutions still have many shortcomings: first, the information parsing process relies heavily on general text summarization or rule-based compression methods, lacks structured summarization strategies for different tool output characteristics, and easily loses key fields; second, the PTT construction process is usually based on local task descriptions, lacks systematic annotation of tactics (Tactics), techniques (Techniques), and sub-techniques (Sub-techniques), resulting in insufficient global planning capabilities and easy path hallucination or repeated execution; third, tool selection is usually limited to a few preset tools, lacking an extensible tool capability library and multi-dimensional selection mechanism; fourth, the task planning process lacks domain knowledge enhancement and fails to fully incorporate security knowledge systems such as the MITRE ATT&CK framework, the CWE vulnerability library, and the OWASP attack pattern, thereby limiting the professionalism and executability of the generated plan. SUMMARY

[0004] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a penetration testing automation method and device based on a large language model and an ATT&CK framework. The present application first structures the multi-source input information and tool output to ensure that key fields are not discarded; then combines retrieval-augmented generation (RAG) technology and a network security knowledge base to provide domain knowledge support for the large language model, thereby generating a penetration testing task tree (PTT) with ATT&CK tactic, technique, and sub-technique annotations; on this basis, the optimal tool is automatically selected through a tool resource library and a multi-dimensional screening mechanism, and execution instructions are generated, and finally the execution results are fed back to the input analysis module to form a self-adaptive and optimized testing closed loop.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a penetration testing automation method based on a large language model and an ATT&CK framework, comprising the following steps:

[0007] Receiving multi-source input information, the multi-source input information includes target information related to penetration testing, constraint conditions, historical data and original output data of at least one penetration testing tool; for the output characteristics of different penetration testing tools, a tool-oriented semantic structured summary mechanism is used, a dedicated analysis template and field mapping rule are defined for each type of tool, a key field non-discardable constraint is set, semantic normalization and conflict resolution are performed on cross-source repeated information, and a standardized evidence object is generated;

[0008] A TTP-aware retrieval enhancement generation module is constructed around the penetration testing task context, a special network security knowledge base covering MITRE ATT&CK framework, vulnerability patterns, tool usage methods and attack chain cases is called, and structured knowledge fragments are obtained through hybrid retrieval and rearrangement strategies;

[0009] The standardized evidence object and the structured knowledge fragment are input into a large language model to generate or incrementally update a penetration testing task tree PTT, each task node of the PTT is bound to a tactic-technology-subtechnology triple of the MITRE ATT&CK framework and task association information, the task path is optimized based on global targets and multi-dimensional indicators, and an operation suppression strategy is introduced to avoid invalid execution;

[0010] A standardized tool repository is constructed, candidate tools are selected from the repository according to the TTP label and task context of the current task node, and the optimal tool is selected according to multi-dimensional indicators, executable instructions and precondition check lists are generated in combination with task information;

[0011] The optimal tool is called in the target environment to execute the task, tool original output information and associated metadata including tool identification, execution environment, running parameters, state indicators and cost risks are collected, and the collected information is standardized and stored persistently;

[0012] The standardized execution results are backflowed, the updated task context is generated by reprocessing, a new round of knowledge enhancement, task planning, tool management and execution process is triggered, the task context is versioned and evolution tracked, and the process is terminated when the penetration target is achieved or the termination condition is met.

[0013] As a preferred technical solution, the penetration testing tool includes Nmap, SQLMap and FFuF;

[0014] For long information processing, combined with field-level filtering, cross-segment information alignment and redundancy merging, semantic normalization and conflict resolution are performed on repeated information across tools and sessions, and confidence labels are added to low-confidence but potentially high-value fields;

[0015] The structured knowledge fragments are obtained through hybrid retrieval and rearrangement strategies, specifically:

[0016] The retrieval process adopts keyword matching and semantic vector hybrid retrieval, and combines TTP matching degree, data source reliability and information timeliness for rearrangement to ensure that the returned content is consistent with the technical context of the current task node;

[0017] After the retrieval results are structured, they are associated with the current task node and used as input for the generation or update of PTT by the large language model, so that task planning depends not only on the currently executed evidence.

[0018] As a preferred technical solution, the TTP-aware retrieval enhancement generation module comprises:

[0019] The knowledge base submodule is used to store domain knowledge related to penetration testing, including tactical, technical and sub-technical information of the MITRE ATT&CK framework, as well as CWE and OWASP vulnerability patterns, common tool usage methods and typical attack chain cases.

[0020] The retrieval submodule is used to construct a retrieval request according to the task context information of the task tree leaf node of the penetration test, and to obtain candidate knowledge fragments from the knowledge base by combining keyword matching and semantic vector retrieval.

[0021] The rearrangement submodule is used to score and rearrange the candidate knowledge fragments according to relevance, reliability, timeliness and TTP semantic consistency, so as to preferentially output structured knowledge fragments that are highly matched with the task context.

[0022] As a preferred technical solution, the repeated operation and high-cost operation suppression strategy sets a budget upper limit or a pre-validation condition for tasks with high repetition or large resource consumption, and automatically degrades to a lightweight verification step when the set condition is met.

[0023] As a preferred technical solution, the tool resource library adopts a unified description rule to standardize the definition of various tools, covering applicable protocols and services, supported TTP labels and adaptable attack stages, as well as required preconditions, parameter templates, output formats, execution costs and time delay information.

[0024] As a preferred technical solution, the metadata includes tool identification information, execution environment information, task association information, execution process parameters, execution status and performance indicators, as well as cost and risk labels.

[0025] The tool identification information includes tool name, version number, and unique ID in the resource library.

[0026] The execution environment information includes execution node IP, operating system type and version, network conditions, and permission level.

[0027] The task association information includes a corresponding TTP label and a task context summary.

[0028] The execution process parameters include actual running command line parameters, start and end times, and execution duration.

[0029] The execution state and performance indicators include running result states, return codes, data packet sending or receiving amounts, and resource occupancy rates.

[0030] The cost and risk labels include resources consumed by execution and security alarm events that can be triggered.

[0031] As a preferred technical solution, the coverage threshold is determined according to a preset penetration testing range and a testing target; the risk threshold is set based on a security level of a target system and a tolerable risk degree; and the cost threshold is demarcated according to a testing resource budget and a time cost requirement.

[0032] In a second aspect, the present application provides a penetration testing automation system based on a large language model and an ATT&CK framework, characterized in that the system is applied to the penetration testing automation method based on the large language model and the ATT&CK framework, and includes a task input analysis module, a knowledge enhancement module, a task planning module, a tool management module, a result collection module, and a closed-loop updating module.

[0033] The task input analysis module is configured to receive multi-source input information, wherein the multi-source input information includes target information related to penetration testing, constraint conditions, historical data, and original output data of at least one penetration testing tool; for output characteristics of different penetration testing tools, a tool-oriented semantic structured summary mechanism is adopted, exclusive analysis templates and field mapping rules are defined for each type of tool, key fields are set as non-removable constraints, semantic normalization and conflict resolution are performed on cross-source repeated information, and a standardized evidence object is generated.

[0034] The knowledge enhancement module is configured to construct a TTP-aware retrieval enhancement generation module around a penetration testing task context, call a special network security knowledge base covering MITRE ATT&CK framework, vulnerability patterns, tool usage methods, and attack chain cases, and obtain structured knowledge fragments through hybrid retrieval and rearrangement strategies.

[0035] The task planning module is configured to input the standardized evidence object and the structured knowledge fragments into a large language model to generate or incrementally update a penetration testing task tree (PTT), wherein each task node of the PTT is bound to a tactic-technology-subtechnology triple of the MITRE ATT&CK framework and task association information, a task path is optimized based on global targets and multi-dimensional indicators, and an operation suppression strategy is introduced to avoid invalid execution.

[0036] The tool management module is configured to build a standardized tool library, filter candidate tools from the library according to a TTP label and a task context of a current task node, and select an optimal tool according to multi-dimensional indexes, and generate executable instructions and a pre-condition check list in combination with task information;

[0037] The result collection module is configured to call the optimal tool to perform a task in a target environment, collect tool original output information and associated metadata including tool identification, execution environment, running parameters, state indexes and cost risks, and perform standardized processing and persistent storage on the collected information.

[0038] The closed-loop update module is configured to return the standardized execution result, reprocess the updated task context to trigger a new round of knowledge enhancement, task planning, tool management and execution process, and perform version management and evolution tracking on the task context, and terminate the process when the penetration target is achieved or the termination condition is met.

[0039] In a third aspect, the present application provides an electronic device, which comprises:

[0040] at least one processor; and

[0041] a memory connected to the at least one processor in communication; wherein

[0042] The memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the penetration testing automation method based on the large language model and the ATT&CK framework.

[0043] In a fourth aspect, the present application provides a computer readable storage medium storing a program, and the program is executed by a processor to implement the penetration testing automation method based on the large language model and the ATT&CK framework.

[0044] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0045] 1. The present application introduces a plurality of innovative technologies in the aspects of task input analysis, knowledge enhancement, task planning and tool management, including structured summary and key field non- discardable constraint for tools, TTP-aware knowledge enhancement, global target and constraint planning based on the ATT&CK framework, and unified tool management mechanism with multi-stage adaptation. This scheme can realize semantic fidelity compression and standardized processing of long information in view of the large differences in output formats and the redundancy of information of different penetration testing tools, effectively reduce path illusion and redundant operations in the task planning stage, and realize high coverage and high matching degree in tool calling, thereby significantly improving the executability and rationality of penetration testing.

[0046] 2. The present application is in the execution and feedback link, by collecting execution output information and tool identification, execution environment, running parameters, state indicators, cost and risk metadata, and returning it to the task input analysis module, combining knowledge enhancement and task planning to form a self-adaptive closed-loop optimization mechanism. This mechanism can continuously optimize the task path and tool selection strategy based on real execution feedback, effectively reduce repeated and inefficient operations, improve the efficiency, stability and repeatability of the whole process of penetration testing, and provide efficient, interpretable and auditable technical support for automated penetration testing in complex network environment. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 Flowchart of the penetration testing automation method based on large language model and ATT&CK framework of the embodiments of the present application;

[0049] Figure 2 Block diagram of the penetration testing automation system based on large language model and ATT&CK framework of the embodiments of the present application.

[0050] Figure 3 Structure diagram of the electronic device of the embodiments of the present application. DETAILED DESCRIPTION

[0051] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] In the present application, "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments.

[0053] As Figure 1As shown, the embodiment based on large language model and ATT&CK framework penetration testing automation method can realize intelligent planning of tasks, automatic selection of tools and closed-loop feedback of execution results in the process of penetration testing. The method first analyzes the structured information and tool output, ensures that the key fields are not discarded; then combines the retrieval augmented generation (RAG) technology and the network security knowledge base to provide domain knowledge support for the large language model, so as to generate a penetration testing task tree (PTT) with ATT&CK tactics, techniques and sub-techniques annotation; On this basis, the optimal tool is automatically selected through the tool resource library and multi-dimensional screening mechanism, and the execution instruction is generated, and finally the execution result is returned to the input analysis module to form a self-adaptive optimization test closed loop. Specifically, the steps include:

[0054] Step 1, task input analysis and arrangement.

[0055] Further, step 1 is specifically:

[0056] Receive multi-source input information from users or systems, including target domain name or IP address, test range, environment constraints, historical clues and raw output data of various penetration testing tools (such as Nmap, SQLMap, FFuF, etc.). Due to the significant differences in format structure, information density and redundancy ratio of different tool outputs, the embodiment designs a tool-aware semantic structured summary mechanism (Tool-Aware Schema) to define exclusive parsing templates and field mapping rules for each tool, and accurately extracts host, port, service, version, fingerprint, path and parameter, credential and key echo and other core fields from unstructured or semi-structured long text output. In this process, the key field non-discarding constraint is introduced to ensure that port numbers, version numbers, CVE numbers, key URL paths, authentication information, key echoes and other semantically important elements are forcibly retained in the compression and integration process. For the processing of long information, the embodiment combines field-level filtering, cross-segment information alignment and redundancy merging strategies to perform semantic normalization and conflict resolution on repeated information across tools and sessions, and adds confidence labels to low-confidence but potentially high-value fields for subsequent reasoning model discrimination. The final standardized evidence object not only significantly reduces the storage volume, but also meets the high-precision requirements of subsequent knowledge retrieval and task planning in terms of information density, retrievability and semantic integrity, fundamentally improving the stability and explainability of task planning in complex context environment.

[0057] Step 2, knowledge enhancement.

[0058] Further, step 2 is specifically:

[0059] A TTP-aware retrieval augmentation generation (RAG) module is constructed around the context of the current penetration testing task, especially the leaf nodes of the penetration testing task tree (PTT) and their associated evidence, to enhance the domain knowledge of the large language model, thereby improving its reasoning accuracy and executability in the subsequent task planning stage. The retrieval augmentation generation module performs retrieval augmentation operations on a specialized network security knowledge base that covers tactical, technical, and sub-technical information from the MITRE ATT&CK framework, as well as authoritative resources such as vulnerability patterns from CWE and OWASP, common tool usage methods, and typical attack chain cases. The retrieval process uses a combination of keyword matching and semantic vector retrieval, and combines TTP matching degree, data source credibility, and information timeliness for rearrangement to ensure that the returned content is highly consistent with the technical context of the current task node. After being structured, the retrieval results are associated with the current task node and used as input for the large language model to generate or update the PTT, so that task planning not only relies on the current executed evidence, but also integrates external authoritative knowledge that matches it. Through this TTP-aware knowledge enhancement mechanism, a high correlation knowledge feedback loop is formed from the execution results to the task planning, effectively reducing the path hallucination of the large model in the planning stage and significantly improving the executability of the generated tasks and the rationality of the attack stage.

[0060] Furthermore, the TTP-aware retrieval augmentation generation module in the embodiment further includes a knowledge base submodule, a retrieval submodule, and a rearrangement submodule. The knowledge base submodule is used to centrally store domain knowledge related to penetration testing, including tactical, technical, and sub-technical information from the MITRE ATT&CK framework, vulnerability patterns from CWE and OWASP, usage instructions for common tools, and typical attack chain cases. The retrieval submodule constructs a retrieval request based on the task context information of the PTT leaf node, and uses a combination of keyword matching and semantic vector retrieval to extract candidate knowledge fragments from the knowledge base. The rearrangement submodule scores and prioritizes the candidate knowledge fragments in terms of relevance, credibility, timeliness, and TTP semantic consistency, to ensure that the returned structured knowledge fragments are highly matched with the current task context. The processed structured knowledge fragments and task context information are input into the large language model to generate or update the PTT.

[0061] Step 3, task planning and PTT generation or update.

[0062] Further, step 3 is specifically:

[0063] With the support of knowledge-enhanced results, the large language model generates or incrementally updates the penetration testing task tree (PTT) based on the evidence obtained from the current execution and the structured search results. Each task node is strictly bound to the tactic, technique, and sub-technique triplets in the MITRE ATT&CK framework, and the preconditions, postconditions, associated evidence, and execution state information are recorded in the node. To ensure the stability and traceability of the task tree structure, this method limits updates to only occur at leaf node locations, avoiding unnecessary changes to the global structure. The task planning process is driven by global goals and constraint planning mechanisms, considering multiple dimensions such as the relevance of tasks and overall goals, the gain brought by TTP coverage, execution cost and potential risk, historical success rate, etc. to optimize the generation order and execution path of tasks, and to achieve reasonable switching between different attack stages. In addition, to reduce ineffective or excessively costly repeated execution, this method introduces a repetitive operation and high-cost operation suppression strategy, setting a budget limit or pre-validation condition for tasks with high repetition or large resource consumption (such as brute force cracking), and automatically downgrading to a lightweight verification step if necessary. In this way, the generation and update of the task tree not only reflect the global perspective of the ATT&CK framework, but also incorporate constraint optimization and cost control mechanisms, significantly improving the executability of task planning and the efficiency of the overall penetration process.

[0064] Step 4, tool management and instruction generation.

[0065] Further, step 4 is specifically:

[0066] On the basis of the TTP label of the current task node and the existing task context information, the optimal execution tool is retrieved and selected from the tool repository. The tool repository adopts a unified description rule to standardize the definition of various tools, covering not only applicable protocols and services, supported TTP labels and adaptable attack stages, but also required prerequisites, parameter templates, output formats, execution costs and time delays and other attribute information. Through this unified structure, a large number of tools with diverse functions can be included in the repository, enabling them to cover different technical paths and penetration tasks at different stages under the same tactics. The tool selection process first filters out the candidate tool set that can be directly used under the current situation according to the matching results of the TTP label and the task context information; then, the candidate tools are prioritized according to multiple indicators such as success rate prior, situation matching degree (service, version and related feature consistency), execution cost and historical execution feedback, to ensure that the selected result not only meets the technical adaptability but also meets the efficiency requirement. After the tool is selected, the system combines its parameter template with the current task context information to automatically generate executable commands and parameters, and outputs a prerequisite check list, such as target port connectivity verification, credential validity confirmation, etc. Through the unified tool description and multi-stage adaptation capability, this step realizes the high automation and interpretability of tool selection and instruction generation, providing high coverage and high adaptability of technical support for the whole process of penetration task execution.

[0067] Step 5, execution and result collection.

[0068] Further, step 5 is specifically:

[0069] This step calls the selected tool and runs the corresponding task in the target environment according to the execution plan generated in step 4. The execution module starts the tool according to the task context information and tool parameter specification, and collects the original execution output information during the execution process, including terminal output, network response, file content, log fragments, and other data that can directly reflect the execution results. At the same time, the execution module also generates and records metadata associated with this execution, which is used to describe the context and traceability of the execution process. The metadata includes: tool identification information (tool name, version number, unique ID in the resource library, etc.), execution environment information (execution node IP, operating system type and version, network conditions, permission level, etc.), task association information (corresponding TTP label, task context summary, etc.), execution process parameters (actual running command line parameters, start and end time, execution duration, etc.), execution status and performance indicators (running result status, return code, data packet sending / receiving volume, resource occupancy rate, etc.), and cost and risk labels (resources consumed by execution, possible security alarm events triggered, etc.). These metadata and execution output information are standardized and stored persistently, providing accurate context support for subsequent knowledge enhancement and task planning, and can be used as a basis for strategy optimization and audit traceability. In this way, this step ensures the full-link automation from tool calling to execution information collection, and significantly improves the explainability and decision reliability of the system in complex penetration tasks.

[0070] Step 6, result backflow and closed-loop update.

[0071] Further, step 6 is specifically:

[0072] The execution output information and its associated metadata obtained in step 5 are fed back to the task input parsing and sorting module of step 1. According to the tool-oriented structured summary rules, the information is re-extracted, normalized and standardized in format, and the key task fields (such as port, service, version number, CVE number, key path and parameter, characteristic echo, etc.) are strictly preserved during compression, thereby generating updated task context information. This task context information will be used as input for the next round of knowledge enhancement, triggering the retrieval and rearrangement of the network security knowledge base in step 2 to obtain structured knowledge fragments highly relevant to the latest task status, and generating or optimizing the penetration testing task tree in step 3 by the large language model. In order to ensure the efficiency and robustness of the closed-loop operation, this step will manage the versioning and change tracking of the execution output information during the feedback, record the evolution path of the task context, and perform relevance assessment and priority labeling on the new information to facilitate the prioritized use of high-value information in the planning stage. When the penetration target has reached or met the termination conditions (such as coverage, risk, cost threshold), the process automatically ends; otherwise, it enters a new round of task execution iteration. Through this feedback and update mechanism based on real execution feedback, the system can continuously optimize the task path and tool selection strategy, forming a self-adaptive "analysis-knowledge enhancement-task planning-tool execution-result feedback" closed-loop link, thereby effectively reducing repeated operations and inefficient attempts, and significantly improving the execution efficiency, stability and interpretability of the entire penetration testing process.

[0073] In one aspect, the present application is designed to address the large differences in output formats and the redundancy of different penetration testing tools. A tool-oriented structured summary mechanism and key field non-removable constraint are designed, and combined with TTP-aware knowledge enhancement, the task context is combined with the network security knowledge base containing MITREATT&CK framework, CWE / OWASP vulnerability patterns and tool usage methods, thereby significantly reducing path illusion and redundant operations in the task planning stage, improving the executability and rationality of the penetration testing task. On the other hand, the present application uses a unified description rule to standardize the definition of the tool resource library, covering information such as applicable protocols, services, TTP labels, attack stages and parameter templates. Through a two-layer filtering mechanism of multi-dimensional indicators, the optimal tool is automatically selected and executable instructions with pre-condition checks are generated. At the same time, execution output information and tool identification, execution environment, running parameters, state indicators, cost and risk metadata are collected during the execution and feedback stages, and fed back to the task planning module to form a self-adaptive closed-loop optimization mechanism, effectively improving the efficiency, stability and repeatability of the entire penetration testing process.

[0074] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other orders or simultaneously.

[0075] Based on the same idea as the penetration testing automation method based on a large language model and the ATT&CK framework in the above embodiment, the present application also provides a penetration testing automation system based on a large language model and the ATT&CK framework, which can be used to execute the penetration testing automation method based on a large language model and the ATT&CK framework described above. For the sake of convenience, in the structural schematic diagram of the embodiment of the penetration testing automation system based on a large language model and the ATT&CK framework, only the parts related to the embodiment of the present application are shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and can include more or fewer components than the illustrated, or combine certain components, or different component arrangements.

[0076] Please refer to Figure 2 In another embodiment of the present application, a penetration testing automation system based on a large language model and the ATT&CK framework 100 is provided, which includes a task input analysis module 101, a knowledge enhancement module 102, a task planning module 103, a tool management module 104, a result collection module 105, and a closed-loop update module 106.

[0077] The task input analysis module 101 is configured to receive multi-source input information, wherein the multi-source input information includes target information related to penetration testing, constraint conditions, historical data, and original output data of at least one penetration testing tool; for the output characteristics of different penetration testing tools, a tool-oriented semantic structured summary mechanism is adopted, a dedicated analysis template and field mapping rule are defined for each type of tool, a key field non-discardable constraint is set, semantic normalization and conflict resolution are performed on cross-source repeated information, and a standardized evidence object is generated.

[0078] The knowledge enhancement module 102 is configured to construct a TTP-aware retrieval enhancement generation module around the context of the penetration testing task, call a special network security knowledge base covering the MITRE ATT&CK framework, vulnerability patterns, tool usage methods, and attack chain cases, and obtain structured knowledge fragments through hybrid retrieval and rearrangement strategies.

[0079] The task planning module 103 is used to input the standardized evidence objects and structured knowledge fragments into the large language model to generate or incrementally update the penetration testing task tree PTT. Each task node of the PTT is bound to the tactical-technical-sub-technical triplet of the MITREATT&CK framework and task association information. The task path is optimized based on global goals and multi-dimensional indicators, and an operation inhibition strategy is introduced to avoid ineffective execution.

[0080] The tool management module 104 is used to build a standardized tool resource library. Based on the TTP tag and task context of the current task node, it filters candidate tools from the resource library and sorts them according to multi-dimensional indicators to select the optimal tool. It also generates executable instructions and a checklist of prerequisites by combining task information.

[0081] The result acquisition module 105 is used to call the optimal tool to perform tasks in the target environment, collect the original output information of the tool and related metadata including tool identifier, execution environment, running parameters, status indicators, cost risks, and perform standardized processing and persistent storage of the collected information.

[0082] The closed-loop update module 106 is used to backflow the standardized execution results, reprocess and generate the updated task context, trigger a new round of knowledge enhancement, task planning, tool management and execution process, perform version management and evolution tracking of the task context, and terminate the process when the penetration target is achieved or the termination conditions are met.

[0083] It should be noted that the penetration testing automation system based on the large language model and the ATT&CK framework of the present invention corresponds one-to-one with the penetration testing automation method based on the large language model and the ATT&CK framework of the present invention. The technical features and beneficial effects described in the embodiments of the penetration testing automation method based on the large language model and the ATT&CK framework are applicable to the embodiments of penetration testing automation based on the large language model and the ATT&CK framework. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0084] Furthermore, in the above embodiments of the penetration testing automation system based on the large language model and the ATT&CK framework, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or the convenience of software implementation. That is, the internal structure of the penetration testing automation system based on the large language model and the ATT&CK framework is divided into different program modules to complete all or part of the functions described above.

[0085] Please see Figure 3In one embodiment, an electronic device is provided for implementing a penetration testing automation method based on a large language model and the ATT&CK framework. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a penetration testing automation program 203 based on a large language model and the ATT&CK framework.

[0086] The first memory 202 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as the portable hard drive of the electronic device 200. In other embodiments, the first memory 202 can be an external storage device of the electronic device 200, such as a plug-in portable hard drive, SmartMediaCard (SMC), SecureDigital (SD) card, FlashCard, etc., equipped on the electronic device 200. Furthermore, the first memory 202 can include both internal and external storage units of the electronic device 200. The first memory 202 can be used not only to store application software and various types of data installed on the electronic device 200, such as the code of the penetration testing automation program 203 based on the large language model and ATT&CK framework, but also to temporarily store data that has been output or will be output.

[0087] In some embodiments, the first processor 201 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the first memory 202 and calls data stored in the first memory 202 to perform various functions of the electronic device 200 and process data.

[0088] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device 200, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0089] The penetration testing automation program 203 based on the large language model and ATT&CK framework, stored in the first memory 202 of the electronic device 200, is a combination of multiple instructions. When run in the first processor 201, it can achieve the following:

[0090] The system receives multi-source input information, which includes target information, constraints, historical data, and raw output data of at least one penetration testing tool related to penetration testing. Based on the output characteristics of different penetration testing tools, a tool-oriented semantic structured summary mechanism is adopted to define exclusive parsing templates and field mapping rules for each type of tool, and set constraints that key fields cannot be discarded. Semantic normalization and conflict resolution are performed on cross-source duplicate information to generate standardized evidence objects.

[0091] Around the context of penetration testing tasks, a TTP-aware retrieval enhancement generation module is built, which calls a dedicated cybersecurity knowledge base covering the MITREATT&CK framework, vulnerability patterns, tool usage methods and attack chain cases, and obtains structured knowledge fragments through hybrid retrieval and rearrangement strategies.

[0092] The standardized evidence objects and structured knowledge fragments are input into the large language model to generate or incrementally update the penetration testing task tree (PTT). Each task node of the PTT is bound to the tactical-technical-sub-technical triplet of the MITREATT&CK framework and task association information. The task path is optimized based on global goals and multi-dimensional indicators, and an operation inhibition strategy is introduced to avoid ineffective execution.

[0093] Build a standardized tool resource library, filter candidate tools from the resource library based on the TTP tag and task context of the current task node, sort them by multi-dimensional indicators to select the best tool, and generate executable instructions and a checklist of prerequisites by combining task information.

[0094] In the target environment, the optimal tool is invoked to execute the task, and the original output information of the tool and associated metadata including tool identifier, execution environment, running parameters, status indicators, cost and risk are collected. The collected information is then standardized and persistently stored.

[0095] The standardized execution results are fed back and reprocessed to generate an updated task context, triggering a new round of knowledge enhancement, task planning, tool management and execution processes. The task context is version-managed and its evolution is tracked. The process terminates when the penetration goal is achieved or the termination conditions are met.

[0096] Furthermore, if the modules / units integrated in the electronic device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0097] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A penetration testing automation method based on a large language model and the ATT&CK framework, characterized in that, Includes the following steps: The system receives multi-source input information, which includes target information, constraints, historical data, and raw output data of at least one penetration testing tool related to penetration testing. Based on the output characteristics of different penetration testing tools, a tool-oriented semantic structured summary mechanism is adopted to define exclusive parsing templates and field mapping rules for each type of tool, and set constraints that key fields cannot be discarded. Semantic normalization and conflict resolution are performed on cross-source duplicate information to generate standardized evidence objects. Around the context of penetration testing tasks, a TTP-aware retrieval enhancement generation module is built, which calls a dedicated cybersecurity knowledge base covering the MITREATT&CK framework, vulnerability patterns, tool usage methods and attack chain cases, and obtains structured knowledge fragments through hybrid retrieval and rearrangement strategies. The standardized evidence objects and structured knowledge fragments are input into the large language model to generate or incrementally update the penetration testing task tree (PTT). Each task node of the PTT is bound to the tactical-technical-sub-technical triplet of the MITREATT&CK framework and task association information. The task path is optimized based on global goals and multi-dimensional indicators, and an operation inhibition strategy is introduced to avoid ineffective execution. Build a standardized tool resource library, filter candidate tools from the resource library based on the TTP tag and task context of the current task node, sort them by multi-dimensional indicators to select the best tool, and generate executable instructions and a checklist of prerequisites by combining task information. In the target environment, the optimal tool is invoked to execute the task, and the original output information of the tool and associated metadata including tool identifier, execution environment, running parameters, status indicators, cost and risk are collected. The collected information is then standardized and persistently stored. The standardized execution results are fed back and reprocessed to generate an updated task context, triggering a new round of knowledge enhancement, task planning, tool management and execution processes. The task context is version-managed and its evolution is tracked. The process terminates when the penetration goal is achieved or the termination conditions are met.

2. The penetration testing automation method based on a large language model and the ATT&CK framework as described in claim 1, characterized in that, The penetration testing tools include Nmap, SQLMap, and FCuF; When processing long information, strategies such as field-level filtering, cross-segment information alignment and redundancy merging are combined to perform semantic normalization and conflict resolution on repeated information across tools and sessions, and to add confidence labels to fields with low confidence but potentially high value. The method of obtaining structured knowledge fragments through a hybrid retrieval and rearrangement strategy specifically includes: The retrieval process employs a hybrid approach of keyword matching and semantic vector retrieval, and is further reordered based on TTP matching degree, data source credibility, and information timeliness to ensure that the returned content is consistent with the technical context of the current task node. After the retrieval results are structured, they are associated with the current task node and used as input for the large language model to generate or update the PTT, thus enabling task planning to rely not only on the evidence of the current execution.

3. The penetration testing automation method based on a large language model and the ATT&CK framework as described in claim 1, characterized in that, The TTP-aware retrieval enhancement generation module includes: The knowledge base submodule is used to store domain knowledge related to penetration testing. This knowledge includes tactical, technical and sub-technical information of the MITREATT&CK framework, as well as CWE and OWASP vulnerability patterns, common tool usage methods and typical attack chain cases. The retrieval submodule is used to construct a retrieval request based on the task context information of the leaf nodes of the penetration testing task, and to obtain candidate knowledge fragments from the knowledge base by combining keyword matching and semantic vector retrieval. The reordering submodule is used to score and reorder candidate knowledge fragments according to relevance, credibility, timeliness, and TTP semantic consistency, so as to prioritize the output of structured knowledge fragments that are highly matched with the task context.

4. The penetration testing automation method based on the large language model and ATT&CK framework as described in claim 1, characterized in that, The repetitive and high-cost operation suppression strategy sets a budget limit or pre-verification conditions for tasks with high repetition or high resource consumption, and automatically downgrades them to lightweight verification steps when the set conditions are met.

5. The penetration testing automation method based on a large language model and the ATT&CK framework according to claim 1, characterized in that, The tool resource library uses unified description rules to standardize the definition of various tools, which not only covers applicable protocols and services, supported TTP tags and adaptable attack stages, but also includes the required preconditions, parameter templates, output formats, execution costs and latency information.

6. The penetration testing automation method based on a large language model and the ATT&CK framework according to claim 1, characterized in that, The metadata includes: The metadata includes tool identification information, execution environment information, task association information, execution process parameters, execution status and performance indicators, and cost and risk labels; The tool identification information includes the tool name, version number, and unique ID in the resource library; The execution environment information includes the execution node IP, operating system type and version, network conditions, and permission level; The task association information includes the corresponding TTP tag and task context summary; The execution process parameters include the actual command line parameters, start and end times, and execution duration. The execution status and performance indicators include the running result status, return code, data packet sending or receiving volume, and resource utilization rate. The cost and risk labels include the resources consumed during execution and the security alert events that may be triggered.

7. The penetration testing automation method based on a large language model and the ATT&CK framework as described in claim 1, characterized in that, The coverage threshold is determined based on the preset penetration test range and test objectives; the risk threshold is set based on the security level and tolerable risk level of the target system. The cost threshold is determined based on the test resource budget and time cost requirements.

8. A penetration testing automation system based on a large language model and the ATT&CK framework, characterized in that: The penetration testing automation method based on the large language model and ATT&CK framework, applied to any one of claims 1-7, includes a task input parsing module, a knowledge enhancement module, a task planning module, a tool management module, a result collection module, and a closed-loop update module; The task input parsing module is used to receive multi-source input information, which includes target information, constraints, historical data, and raw output data of at least one penetration testing tool related to penetration testing. Based on the output characteristics of different penetration testing tools, a tool-oriented semantic structured summarization mechanism is adopted to define exclusive parsing templates and field mapping rules for each type of tool, and to set constraints that key fields cannot be discarded. Semantic normalization and conflict resolution are performed on cross-source duplicate information to generate standardized evidence objects. The knowledge enhancement module is used to build a TTP-aware retrieval enhancement generation module around the penetration testing task context. It calls a dedicated cybersecurity knowledge base covering the MITREATT&CK framework, vulnerability patterns, tool usage methods, and attack chain cases, and obtains structured knowledge fragments through a hybrid retrieval and rearrangement strategy. The task planning module is used to input the standardized evidence objects and structured knowledge fragments into the large language model to generate or incrementally update the penetration testing task tree (PTT). Each task node of the PTT is bound to the tactical-technical-sub-technical triplet of the MITREATT&CK framework and task association information. The task path is optimized based on global goals and multi-dimensional indicators, and an operation inhibition strategy is introduced to avoid ineffective execution. The tool management module is used to build a standardized tool resource library. Based on the TTP tag and task context of the current task node, it filters candidate tools from the resource library and sorts them according to multi-dimensional indicators to select the optimal tool. It also generates executable instructions and a checklist of prerequisites by combining task information. The result acquisition module is used to call the optimal tool to execute the task in the target environment, collect the original output information of the tool and related metadata including tool identifier, execution environment, running parameters, status indicators, cost risks, and perform standardized processing and persistent storage of the collected information. The closed-loop update module is used to backflow the standardized execution results, reprocess and generate the updated task context, trigger a new round of knowledge enhancement, task planning, tool management and execution process, perform version management and evolution tracking of the task context, and terminate the process when the penetration target is achieved or the termination conditions are met.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor to enable the at least one processor to perform the penetration testing automation method based on the large language model and ATT&CK framework as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the penetration testing automation method based on the large language model and ATT&CK framework as described in any one of claims 1-7.

Images