Information collection method, device, computer equipment and medium in penetration testing

Abstract

Embodiments of the present application provide a method and device for information collection in penetration testing, computer equipment and medium, which belong to the technical field of security testing, and the method comprises the following steps: receiving an information collection instruction in natural language form, identifying the scene type of the penetration testing information collection scene corresponding to the information collection instruction based on a large language model, decomposing the information collection instruction into multiple sub-intention data based on the large language model, and binding each sub-intention data with the scene type corresponding to the information collection instruction; dynamically generating a sub-task execution command corresponding to each sub-intention data according to each sub-intention data and the bound scene type; and dynamically selecting a task scheduling strategy that is adapted to each sub-task execution command to call a corresponding function module to complete information collection according to the scene type included in each sub-task execution command. The scheme improves the efficiency, accuracy, flexibility and adaptability of information collection.

Description

Technical Field

[0001] This invention relates to the field of security testing technology, and in particular to an information collection method, apparatus, computer equipment, and medium for penetration testing. Background Technology

[0002] With the rapid development of information technology, computer systems and network infrastructure have become the core platforms for enterprise operations, service organizations, and personal communication. To ensure the security of these systems, information security technology is receiving increasing attention, and one important security testing method is penetration testing.

[0003] Penetration testing is a proactive testing method that assesses system security by simulating hacker attacks. It is typically conducted by professional security personnel who, from an attacker's perspective, perform operations such as vulnerability probing, privilege escalation, and obtaining sensitive information from the target system. In penetration testing, "information gathering" is the first and crucial step in the testing process. Its purpose is to obtain as much basic information as possible about the target system to support subsequent attack simulations. Common information gathering tasks include: identifying target IP ranges, detecting open ports, collecting subdomains, identifying web application fingerprints, and enumerating directory structures.

[0004] In existing technologies, information gathering primarily relies on penetration testers manually or via scripts calling multiple specialized tools, such as:

[0005] Use nmap or masscan to perform port scanning;

[0006] Use tools such as subfinder and amass to detect subdomains;

[0007] Use tools such as whatweb and httpx to identify web services;

[0008] Use tools such as dirsearch and gobuster to scan directories.

[0009] While the aforementioned tools are powerful, they are mostly command-line based, making them relatively difficult to use. They require manual intervention to determine parameters, execute sequences, and integrate results. Especially when facing complex objectives or large-scale testing, manually orchestrating tasks is not only time-consuming and labor-intensive, but also prone to information omissions due to lack of experience.

[0010] In addition, while automated scripts can improve collection efficiency to some extent, they lack flexibility and are difficult to dynamically respond to users' temporary needs (such as "scanning ports first, and then conducting in-depth probing of HTTP services").

[0011] Currently, the widely adopted penetration testing information gathering methods in the industry are typically based on integrated tool platforms or script automation frameworks. The typical technical implementation path of this method is as follows:

[0012] Manual configuration of task parameters: Testers manually set information such as scan type, target domain name / IP, scan port range, and fingerprint recognition method based on the penetration target;

[0013] Script or platform-based scanning tools: Use a set of pre-defined scripts (such as Python / Bash), or chain multiple information gathering tools within an integrated platform (such as goby, Nmap, etc.) to perform the following tasks:

[0014] Subdomain detection (e.g., using Sublist3r, amass)

[0015] Port scanning (such as Nmap, masscan)

[0016] IP address geolocation identification (e.g., ipwhois, online API)

[0017] Website directory scanning (such as dirsearch, gobuster)

[0018] Web application fingerprinting (such as WhatWeb, Wappalyzer)

[0019] Results Compilation and Recording: After completing various scans, the testers manually collect, filter, and integrate the scan results to form a preliminary information collection report.

[0020] The core characteristics of this type of technical solution are:

[0021] It is based on manual configuration, tool combination script execution, and manual summarization of scan results.

[0022] Although such methods are widely used in professional teams and the tools are mature and scalable, they are essentially still a semi-automated approach that heavily relies on manual operation. While there has been some progress in automation, the following significant drawbacks remain:

[0023] 1. High reliance on manual configuration, high barrier to entry for users.

[0024] This method relies on testers being familiar with various information collection tools, including their parameters, usage methods, and applicable scenarios.

[0025] For example, to perform a port scan task, testers need to know how to use the Nmap command line, how to set the scan type (such as SYN scan, UDP scan), and how to limit the scan scope.

[0026] It is evident that the numerous tools and complex commands make it difficult for non-professionals to complete the configuration independently, thus limiting the applicability of this method to people with low technical skills.

[0027] 2. Lack of unified task scheduling and context understanding capabilities

[0028] This technology relies on scripts or platforms to manually configure the task execution order and data flow, lacking an intelligent context processing mechanism.

[0029] For example, if a subdomain probe returns multiple results, subsequent port scans require manual extraction of these subdomains and reconfiguration of the target, lacking an automatic routing mechanism.

[0030] Therefore, this technology lacks the ability to link and respond between modules, resulting in low efficiency and the inability to automatically connect task contexts.

[0031] 3. Limited automation and fragmented information collection process.

[0032] Despite the use of automated tools, the entire information collection chain still requires human intervention, for example:

[0033] Task switching (first subdomain, then port, then directory) requires manual intervention;

[0034] The results need to be manually compared and categorized.

[0035] This type of operation results in penetration testers spending a lot of time on non-core tasks, such as file cleaning, data format conversion, and tool compatibility.

[0036] This fragments the entire information collection process, lacks centralized control, and makes it difficult to meet the actual needs of efficient, batch execution.

[0037] In summary, existing traditional information gathering methods have significant shortcomings in terms of automation, ease of use, modularity, and intelligence. This is because the aforementioned technical approaches employ a combination of manual configuration, tools, and static processes, resulting in low efficiency and an inability to meet the increasing demands of penetration testing. Summary of the Invention

[0038] In view of this, embodiments of the present invention provide an information collection method for penetration testing to solve the technical problems of low efficiency, low usability, and low intelligence in information collection in the prior art. The method includes:

[0039] The system receives information collection instructions in natural language form, identifies the scenario type of the penetration test information collection scenario corresponding to the information collection instructions based on a large language model, and decomposes the information collection instructions into multiple sub-intent data based on the large language model, and binds each sub-intent data to the scenario type corresponding to its information collection instructions.

[0040] Based on each sub-intent data and the bound scenario type, dynamically generate the sub-task execution command corresponding to each sub-intent data;

[0041] Based on the scenario type included in each subtask execution command, a task scheduling strategy adapted to each subtask execution command is dynamically selected, and based on the selected task scheduling strategy, the corresponding functional module is called to complete information collection according to the subtask execution command.

[0042] This invention also provides an information collection device for penetration testing, addressing the technical problems of low efficiency, low usability, and low intelligence in information collection in the prior art. The device includes:

[0043] The intent recognition module is used to receive information collection instructions in natural language form, identify the scenario type of the penetration test information collection scenario corresponding to the information collection instructions based on a large language model, and decompose the information collection instructions into multiple sub-intent data based on the large language model, and bind each sub-intent data to the scenario type corresponding to its information collection instructions.

[0044] The command generation module is used to dynamically generate a subtask execution command corresponding to each sub-intent data based on each sub-intent data and the bound scenario type;

[0045] The information collection module is used to dynamically select a task scheduling strategy that is compatible with the scenario type included in each subtask execution command, and to call the corresponding functional module to complete the information collection based on the selected task scheduling strategy and the subtask execution command.

[0046] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned information collection methods in penetration testing, thereby solving the technical problems of low efficiency, low usability, and low intelligence in information collection in the prior art.

[0047] This invention also provides a computer-readable storage medium storing a computer program that executes any of the information collection methods described above in penetration testing, in order to solve the technical problems of low efficiency, low usability, and low intelligence in information collection in the prior art.

[0048] Compared with the prior art, the beneficial effects achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: introducing LLM (Large Language Model) into the information collection stage of penetration testing, through the natural language parsing capability of the large language model, information collection instructions can be decomposed into multiple sub-intent data and the scenario type of the penetration testing information collection scenario corresponding to the information collection instructions can be identified, enabling non-professional users to drive information collection tasks in a natural language manner such as dialogue, thereby achieving the direct effect of reducing the technical threshold and improving ease of use; at the same time, in the process of generating sub-task execution commands, each sub-intent can be dynamically generated based on the sub-intent data and the bound scenario type. The system generates and schedules sub-tasks corresponding to the graph data, and selects a task scheduling strategy that matches the scenario type for each sub-task execution command during the task scheduling phase. This strategy calls the corresponding functional modules to complete information collection, achieving automated and intelligent generation and scheduling of sub-task execution commands. This improves the automation and efficiency of information collection. Compared with traditional semi-automatic methods that rely on manual intervention, this application can further improve the efficiency of information collection. It implements a scenario division mechanism based on scenario type, and dynamically generates and schedules sub-task execution commands based on scenario type. This helps to improve the rationality of command generation and the matching of dynamic task scheduling strategies, thereby improving the accuracy, flexibility, and adaptability of information collection. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of an information collection method in penetration testing provided by an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the principle architecture of an information collection method in penetration testing provided by an embodiment of the present invention;

[0052] Figure 3 This is a flowchart illustrating the principle of an information collection method in penetration testing provided by an embodiment of the present invention;

[0053] Figure 4 This is a structural block diagram of a computer device provided in an embodiment of the present invention;

[0054] Figure 5 This is a structural block diagram of an information collection device for penetration testing provided in an embodiment of the present invention. Detailed Implementation

[0055] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0056] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] In this embodiment of the invention, an information collection method for penetration testing is provided, such as... Figure 1 As shown, the method includes:

[0058] Step S101: Receive information collection instructions in natural language form, identify the scenario type of the penetration test information collection scenario corresponding to the information collection instructions based on the large language model, and decompose the information collection instructions into multiple sub-intent data based on the large language model, and bind each sub-intent data to the scenario type corresponding to its information collection instructions;

[0059] Step S102: Based on each sub-intent data and the bound scene type, dynamically generate the sub-task execution command corresponding to each sub-intent data;

[0060] Step S103: Based on the scenario type included in each subtask execution command, dynamically select a task scheduling strategy that adapts to each subtask execution command, and call the corresponding functional module to complete information collection based on the selected task scheduling strategy and the subtask execution command.

[0061] Depend on Figure 1As shown in the flowchart, in this embodiment of the invention, LLM (Large Language Model) is introduced into the information gathering stage of penetration testing. Through the natural language parsing capabilities of the LLM, information gathering instructions can be broken down into multiple sub-intent data and the scenario type of the penetration testing information gathering scenario corresponding to the information gathering instructions can be identified. This allows non-professional users to drive information gathering tasks using natural language such as dialogue, thereby directly lowering the technical threshold and improving usability. Simultaneously, in the process of generating sub-task execution commands, the sub-task execution command corresponding to each sub-intent data can be dynamically generated based on the sub-intent data and the bound scenario type. This invention enables automated and intelligent generation and scheduling of subtask execution commands, improving the automation and efficiency of information collection. Compared to traditional semi-automatic methods relying on manual intervention, this application further enhances the efficiency of information collection. It implements a scenario-based segmentation mechanism, dynamically generating and scheduling subtask execution commands based on scenario types, which improves the rationality of command generation and the matching of dynamic task scheduling strategies, thereby enhancing the accuracy, flexibility, and adaptability of information collection.

[0062] In specific implementation, such as Figure 2 , 3 As shown, the principle architecture of the information collection method in the above penetration test can include an interaction module, an LLM module, an instruction engine, a scheduling engine, a port scanning module, a subdomain detection module, an IP address collection module, a directory scanning module, a Web application fingerprinting module, an API interface collection module, and a passive collection module. It aims to achieve a high degree of automation and closed-loop in the penetration test information collection process through natural language parsing, user intent decomposition, automatic command generation and scheduling execution, thereby reducing operational complexity and improving task execution efficiency.

[0063] In practice, the process of receiving information collection instructions in natural language can be achieved through an interactive module. This module enables human-computer interaction between the user and the system. Users can input their penetration testing information collection needs (i.e., the aforementioned information collection instructions) in natural language through the interactive module, which is configured with an interactive window. These input information collection needs can cover various test targets and scope descriptions, such as specifying domain names, IP ranges, open port ranges, directory scan depths, etc.

[0064] As a concrete implementation of the interaction module, its chat interface allows users to submit their requests in natural language. These user-submitted natural language commands (i.e., the aforementioned information collection commands) are encapsulated into structured requests and sent to the LLM module via API or an internal message queue for subsequent use. Simultaneously, the chat interface also displays the data results of the information collection tasks performed in the background. After the backend completes its execution, it returns the results to the interaction module in JSON format, including information such as a list of scanned subdomains and port open status. The interaction module automatically formats the data according to its data type and displays it in the chat interface.

[0065] In practical implementation, existing traditional information gathering methods only support parameter-level instruction configuration and lack the ability to translate user intent into operational tasks. This prevents users from driving information gathering by describing tasks using natural language or business context. To address this issue, this application introduces a large language model into the information gathering phase of penetration testing. Figure 2 , 3 The LLM module shown.

[0066] Specifically, the LLM module integrates the semantic parsing capabilities of the Large Language Model (LLM). The LLM module's input interface receives raw natural language data packets (i.e., the aforementioned information collection instructions) transmitted by the interaction module. These data packets contain user instruction text and optional contextual information. By calling the LLM module, combined with a pre-built Prompt template library and contextual history information, the module identifies and decomposes the user's natural language instructions into task intent. Using an inference strategy, it breaks down the overall task (i.e., the aforementioned information collection instructions) into several independent user sub-intent data. Simultaneously, based on its natural language semantic understanding capabilities, the LLM module performs scene recognition on the input content and context, determining the scene type of the penetration testing information collection scenario to which each sub-intent belongs. The identified scene type is then bound to the sub-intent data to guide subsequent instruction generation and functional module invocation, ensuring a high degree of adaptation between the generated commands and the scene type. This step ensures comprehensive and accurate parsing of the received intent, avoiding task execution deviations caused by natural language ambiguity, and introduces intelligent scene-aware features, providing a foundation for dynamic instruction generation.

[0067] For example:

[0068] The user's input in natural language was: "Please scan all subdomains of example.com, then scan ports 80 and 443 for each subdomain, and detect the web application fingerprints of these sites." The LLM module, during parsing, performed the following steps:

[0069] General intent: "Please help me scan all subdomains of example.com first, then collect IP information based on the subdomains, and perform a port scan (scenario: specific target collection)."

[0070] The overall intent will be broken down into the following sub-intent data, and all sub-intent data of the same overall intent will be labeled with a unified scenario type (that is, multiple sub-intent data obtained from the information collection instruction will be bound to the same scenario type):

[0071] Ziyi Figure 1 : Perform a task to collect subdomains of the target example.com (scenario: specific target collection scenario);

[0072] Ziyi Figure 2 Collect IP information based on subdomains of example.com (Scenario: Target-specific data collection scenario);

[0073] Ziyi Figure 3 : Perform port scanning tasks on the collected IPs (scenario: specific target collection scenario).

[0074] In practice, the types of scenarios for penetration testing information collection can include:

[0075] Comprehensive data collection scenario: This refers to the collection of all assets and information related to the target, with a wide coverage, suitable for initial surveys and panoramic analysis.

[0076] Partial data collection scenario: This refers to collecting information on a specific type of asset to focus on key assets.

[0077] Targeted information gathering scenarios: This refers to information gathering targeting specific domains, IPs, services, or business functions, and is applicable to known targets.

[0078] Passive collection scenario: This refers to collecting information only through public resources or in a way that does not require direct interaction with the target system, in order to reduce risk exposure and obtain asset information without triggering alarms. The collection results are limited, but the concealment is extremely strong.

[0079] Rapid data collection scenario: This prioritizes speed and aims to quickly acquire high-value information for rapid evaluation.

[0080] Continuous data collection scenario: This refers to periodic or long-term data collection to track new assets and changes.

[0081] Target association collection scenario: This involves analyzing subsidiaries, partners, and outsourced teams to find breakthroughs for the main target.

[0082] Industry-specific data collection scenarios: These are data collection strategies tailored to specific industries or business scenarios, such as data collection in industries with industry characteristics, such as finance, power, and healthcare.

[0083] The identified scene information is bound to sub-intent data to guide subsequent command generation and function module invocation, ensuring that the generated commands are highly adapted to the scene. This step ensures that the received intents can be comprehensively and accurately parsed, avoiding task execution deviations caused by natural language ambiguity, while introducing intelligent scene-aware features to provide a foundation for dynamic command generation.

[0084] In practical implementation, to generate instructions more reasonably, accurately, and flexibly, this embodiment proposes a method for generating corresponding subtask execution commands for each sub-intent data. For example,

[0085] Based on each sub-intent data and the bound scenario type, generate a subtask execution command corresponding to each sub-intent data, including:

[0086] Semantic recognition and relation extraction are performed on each of the sub-intent data to obtain at least one sub-task, task objective and related constraints;

[0087] The corresponding functional module is determined based on the scenario type, the type of the subtask, and the task objective of each sub-intent data binding;

[0088] Based on the scenario type and related constraints of each sub-intent data binding, the operating parameters of the functional module are generated;

[0089] The subtask execution command is generated based on the running parameters. The subtask execution command includes the task ID, scenario type, and running parameters.

[0090] Specifically, during the deep semantic parsing process for each of the sub-intent data, semantic recognition and relation extraction are performed to obtain at least one specific sub-task, task target (such as target domain name, port range, scan depth, etc.) and related constraints; all the parsed sub-tasks, scene tags, and user constraints are mapped to a predefined JSON data structure, and the generated JSON structured data is returned to the instruction engine through the API interface to enable dynamic generation of instructions.

[0091] Specifically, during the generation of subtask execution commands in the instruction engine, after receiving sub-intent data and its associated scene information, the instruction engine automatically selects the appropriate functional module through an algorithm, dynamically adjusts the parameters of the functional module to adapt to the information collection needs of the scene type, and finally forms an executable command. This step includes the following:

[0092] (1) Parameter generation input

[0093] Sub-intention data (e.g., sub-task type and target object, such as performing a port scan on a certain IP or performing a subdomain probe on a certain domain).

[0094] Scenario types (e.g., comprehensive collection scenario, partial collection scenario, specific target collection scenario, passive collection scenario, rapid collection scenario, continuous collection scenario, target-related collection scenario, industry-specific collection scenario, etc.)

[0095] User constraints (i.e., the aforementioned constraints, such as scan rate, target range, time window, port range, and other user-defined constraints)

[0096] (2) Parameter generation logic

[0097] Rule mapping: Dynamically selects the corresponding functional modules for subtasks based on scenario type, subtask type, and task objective. Through this rule mapping matching process, each subtask can automatically select the functional module that best matches its scenario type and execution strategy, providing a quantifiable and controllable foundation for subsequent parameter generation, command construction, and strategy-driven scheduling and execution. Functional modules include, for example... Figure 2 The module shown includes various functional modules such as port scanning, subdomain, directory detection, application detection, API collection, passive collection, and web application fingerprinting. Each module encapsulates corresponding tools that can execute specific tasks based on tasks and command-line instructions issued by the scheduling engine.

[0098] Dynamic adaptation: Based on the defined functional modules, combined with resource status, scenario type and user constraints, the operating parameters such as scanning rate, number of threads, and scanning depth are dynamically adjusted to ultimately generate parameters suitable for this subtask.

[0099] For example, in a comprehensive data collection scenario: the port range is filled with 1-65535, and the number of threads is set to a medium level to balance speed and risk.

[0100] Rapid data collection scenario: Fill the port range with commonly used ports such as 80, 443, and 22, and increase the number of threads to improve scanning speed;

[0101] Passive data collection scenario: Choose low-risk tools and open-source information to avoid directly scanning targets.

[0102] At the same time, user constraints (such as scan rate and time window) are taken into account, and the number of threads, scan depth, and timeout parameters are dynamically adjusted.

[0103] (3) Command generation

[0104] The generated runtime parameters are combined with the tool command template to form the final executable subtask execution command; for example:

[0105] Subdomain enumeration command: subfinder -d example.com;

[0106] Port scanning command: nmap -Pn -p 80,443 192.168.1.1;

[0107] Web fingerprinting command: whatweb https: / / sub.example.com.

[0108] The generated subtask execution commands can also be input into a large language model, and the AI's capabilities can be used to automatically verify the rationality and legality of the generated commands, which can solve the problems of incorrect command parameter configuration and execution failure in traditional penetration testing tools.

[0109] (4) Output and Binding:

[0110] Each subtask execution command is bound to a corresponding sub-intent and scene type, facilitating scheduling module invocation and task management. The generated subtask execution commands are packaged into execution task objects, including task ID, dependencies, scene tags, and parameter information, and output to the scheduling engine for subsequent policy-based scheduling and parallel / sequential execution.

[0111] In practical implementation, to ensure the correct execution order of instructions, the dependencies between subtasks need to be considered during instruction generation. For example, for at least one subtask, the dependencies between subtasks are identified. For subtasks with dependencies, execution commands are generated according to the order of the dependencies, and these execution commands include the dependencies. Through dependency-aware processing, for subtasks with dependencies, commands for critical dependent subtasks are generated first; for example, IP collection is performed after subdomain enumeration is completed. An internal dependency graph (DAG) (i.e., a graph recording the dependencies between subtasks) can be invoked to ensure that commands are executed in the correct order and to support the parallel execution of commands for subtasks without dependencies.

[0112] The instruction engine can dynamically generate adaptive subtask execution commands for each subtask based on different scenario types, sub-intents, and user constraints, thereby automating, intelligentizing, and increasing the efficiency of penetration testing information collection tasks.

[0113] In practical implementation, to further improve the efficiency and flexibility of information collection, a task scheduling strategy is proposed to dynamically select an appropriate task scheduling strategy based on the scenario type included in each subtask execution command, including:

[0114] If the scenario type is rapid data collection, choose a speed-first strategy to quickly acquire high-value information. For example, in a subdomain collection task, under the speed-first strategy, prioritize calling the asset mapping platform to collect subdomains instead of executing subdomain brute-force tasks. If the scenario type is comprehensive data collection, choose a breadth-first strategy. To ensure the most comprehensive coverage of information collection, it is necessary to continuously expand the breadth of information collection. If the scenario type is passive data collection, the system automatically adopts a stealth-first strategy. The purpose of passive data collection is to avoid triggering alarms and interception by security devices, so tasks that initiate detection and scanning behaviors are not called, thus stealth is prioritized. If the scenario type is partial data collection and specific target data collection, adopt a depth-first strategy to conduct in-depth data collection on limited assets. If the scenario type is continuous data collection, adopt a breadth-first strategy, which is generally suitable for understanding overall asset changes. If the scenario type is target-related data collection, the system automatically adopts a depth-first strategy to deeply collect related companies and suppliers from web pages, enterprise relationships, and other content. If the scenario type is industry-specific data collection, adopt a risk-first strategy to prioritize collecting risks or vulnerabilities in specific software within that industry to find breakthroughs.

[0115] In specific implementation, such as Figure 2 , Figure 3 As shown, the scheduling engine selects an appropriate task scheduling strategy for each subtask execution command. The scheduling engine receives executable subtask execution commands generated by the instruction engine and dynamically arranges the task execution order based on the subtask's scenario type, dependencies, and preset scheduling strategies. Simultaneously, it calls the corresponding functional modules to complete information collection. The scheduling engine selects the task scheduling strategy based on the task's scenario type. For example, if the scenario type is "rapid collection," the system automatically applies a speed-first strategy to quickly acquire high-value information. For instance, in a subdomain collection task, under the speed-first strategy, the asset mapping platform is prioritized for collecting subdomains, rather than executing a subdomain brute-force task. If the scenario type is "comprehensive collection," the system can adopt a breadth-first strategy, simultaneously scheduling subdomain detection, port scanning, and directory scanning tasks, allowing multiple tools to execute in parallel to ensure coverage.

[0116] In practice, task scheduling strategies include: breadth-first strategy, which prioritizes the allocation of execution among multiple different subtasks; depth-first strategy, which prioritizes execution along a chain of subtasks; covert priority strategy, which prioritizes the selection of passive, low-interaction, or low-risk subtasks for execution; risk-first strategy, which prioritizes scheduling tasks for risk collection and detection; and speed-first strategy, which prioritizes the execution of tasks that take less time or can quickly yield results.

[0117] In addition to task scheduling strategies, the scheduling engine also needs to resolve dependencies between tasks when scheduling tasks. For example, IP collection depends on subdomain discovery results, and Web fingerprinting depends on port scanning results and subdomain discovery results. Based on these dependencies, the scheduling engine schedules each subtask to execute the collection task according to the task scheduling strategy. That is, the scheduling engine prioritizes and sorts the subtask execution commands based on the selected task scheduling strategy and dependencies, and then calls the corresponding functional modules to execute the information collection task according to the sorting results. High-priority subtask execution commands are executed first, while low-priority subtask execution commands are scheduled for parallel execution based on system resource availability. During execution, the scheduling engine can also monitor the status of subtask execution commands and collect execution results for subsequent result analysis and scheduling optimization.

[0118] Based on policy-driven and dependency-aware principles, the scheduling engine dynamically schedules task execution. For example, under a breadth-first strategy, if information about domains a.com, b.com, and c.com needs to be collected, the scheduling engine will first utilize all subdomain collection modules to collect subdomains. Then, based on all subdomains, the scheduling engine will simultaneously launch IP collection and port scanning tasks, executing them in parallel. Under a depth-first strategy, the scheduling engine can complete the a.com collection task first, and then execute b.com and c.com, ensuring the continuity of the task chain.

[0119] The scheduling engine is a task scheduling engine driven by task scheduling strategies. It can dynamically adjust the execution order of subtask commands according to different scenario types and requirements, and achieve comprehensive optimization of efficiency, concealment and comprehensiveness while ensuring the correctness of dependencies.

[0120] In practice, based on the scheduling engine's allocation and arrangement of subtask execution commands, the subtask execution commands are executed and the results are obtained. The scheduling engine can also perform deduplication (e.g., deduplication of identical IPs), correlation analysis (e.g., binding port open information to domain names), and format conversion (unifying to JSON) on the obtained multi-source results. The interactive module is then notified to provide a visual representation.

[0121] The aforementioned information collection method in penetration testing achieves fully automated closed-loop operation of the penetration testing information collection process through unified and intelligent control logic. It solves the problems of excessive manual intervention, cumbersome configuration, low automation, and poor task flexibility in the traditional penetration testing information collection process, significantly improving the efficiency, accuracy, and ease of use of information collection, and has good prospects for practical application.

[0122] In this embodiment, a computer device is provided, such as... Figure 4As shown, it includes a memory 401, a processor 402, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the information gathering methods in the penetration test described above.

[0123] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0124] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that performs any of the information gathering methods in the penetration test described above.

[0125] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media does not include transient media, such as modulated data signals and carrier waves.

[0126] Based on the same inventive concept, this invention also provides an information gathering device for penetration testing, as described in the following embodiments. Since the principle of the information gathering device in penetration testing is similar to that of the information gathering method in penetration testing, the implementation of the information gathering device in penetration testing can refer to the implementation of the information gathering method in penetration testing, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0127] Figure 5 This is a structural block diagram of an information collection device in a penetration test according to an embodiment of the present invention, such as... Figure 5 As shown, it includes:

[0128] The intent recognition module 501 (which can be implemented through a large language model) is used to receive information collection instructions in natural language form, identify the scenario type of the penetration test information collection scenario corresponding to the information collection instructions based on the large language model, and decompose the information collection instructions into multiple sub-intent data based on the large language model, and bind each sub-intent data to the scenario type corresponding to its information collection instructions.

[0129] The command generation module 502 (which can be implemented through the command engine or its corresponding functional unit) is used to dynamically generate a subtask execution command corresponding to each sub-intent data according to each sub-intent data and the bound scene type.

[0130] The information collection module 503 (which can be implemented through the scheduling engine or its corresponding functional unit) is used to dynamically select a task scheduling strategy that is compatible with each subtask execution command according to the scenario type included in each subtask execution command, and to call the corresponding functional module to complete information collection based on the selected task scheduling strategy and the subtask execution command.

[0131] In one embodiment, the command generation module is configured to perform semantic recognition and relation extraction on each of the sub-intent data to obtain at least one sub-task, task objective, and related constraints; determine a corresponding functional module based on the scenario type bound to each of the sub-intent data, the type of the sub-task, and the task objective; generate operating parameters for the functional module based on the scenario type bound to each of the sub-intent data and the related constraints; and generate a sub-task execution command based on the operating parameters, wherein the sub-task execution command includes a task ID, scenario type, and operating parameters.

[0132] In one embodiment, the command generation module is used to dynamically generate the number of threads, scan depth, and timeout parameters of the functional module based on the scenario type and related constraints of each sub-intent data binding.

[0133] In one embodiment, the command generation module is further configured to identify dependencies between at least one subtask, and generate subtask execution commands corresponding to the subtasks with dependencies according to the order of the dependencies, wherein the subtask execution commands include the dependencies.

[0134] In one embodiment, the scenario types include: comprehensive collection scenario, partial collection scenario, specific target collection scenario, rapid collection scenario, passive collection scenario, continuous collection scenario, target-related collection scenario, and industry-specific collection scenario.

[0135] In one embodiment, the task scheduling strategy includes: a breadth-first strategy, which prioritizes the allocation of execution among multiple different subtasks; a depth-first strategy, which prioritizes the execution along a chain of subtasks; a covert-first strategy, which prioritizes the execution of passive, low-interaction, or low-risk subtasks; a risk-first strategy, which prioritizes the scheduling of tasks for risk collection and detection; and a speed-first strategy, which prioritizes the execution of tasks that take less time or can quickly yield results.

[0136] In one embodiment, the information collection module is configured to select a speed-first strategy if the scenario type is a rapid collection scenario; a breadth-first strategy if the scenario type is a comprehensive collection scenario; a concealment-first strategy if the scenario type is a passive collection scenario; a depth-first strategy if the scenario type is a partial collection and specific target collection scenario; a depth-first strategy if the scenario type is a target-related collection scenario; and a risk-first strategy if the scenario type is an industry-specific collection scenario.

[0137] The embodiments of this invention achieve the following technical effects: By introducing LLM (Large Language Model) into the information gathering stage of penetration testing, the natural language parsing capability of LLM can decompose information gathering instructions into multiple sub-intent data and identify the scenario type of the penetration testing information gathering scenario corresponding to the information gathering instructions. This allows non-professional users to drive information gathering tasks in a natural language manner, such as dialogue, thereby directly reducing the technical threshold and improving usability. Simultaneously, in the process of generating sub-task execution commands, the sub-task execution command corresponding to each sub-intent data can be dynamically generated based on the sub-intent data and the bound scenario type. In the task scheduling stage, a task scheduling strategy adapted to each sub-task execution command is selected based on the scenario type to call the corresponding functional module to complete information gathering. This achieves automated and intelligent generation and scheduling of sub-task execution commands, improving the automation and efficiency of information gathering. Compared with traditional semi-automatic methods relying on manual intervention, this application can further improve the efficiency of information gathering. The scenario-based scenario division mechanism and the dynamic generation and scheduling of sub-task execution commands based on scenario types help improve the rationality of command generation and the matching of dynamic task scheduling strategies, thereby enhancing the accuracy, flexibility, and adaptability of information gathering.

[0138] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of information gathering in penetration testing, characterized by, The method comprises the following steps: receiving an information collection instruction in natural language form, identifying a scene type of a penetration testing information collection scene corresponding to the information collection instruction based on a large language model, and decomposing the information collection instruction into a plurality of sub-intention data based on the large language model, and binding each sub-intention data to the scene type corresponding to the information collection instruction; dynamically generating a sub-task execution command corresponding to each sub-intention data according to each sub-intention data and the bound scene type; dynamically selecting a task scheduling strategy suitable for each sub-task execution command according to the scene type included in each sub-task execution command, and calling a corresponding function module to complete information collection based on the selected task scheduling strategy according to the sub-task execution command; generating a sub-task execution command corresponding to each sub-intention data according to each sub-intention data and the bound scene type, comprising: performing semantic recognition and relation extraction on each sub-intention data to obtain at least one sub-task, a task target and related constraint conditions; determining a corresponding function module according to the scene type bound to each sub-intention data, the type of the sub-task and the task target; generating running parameters of the function module according to the scene type bound to each sub-intention data and the related constraint conditions; generating the sub-task execution command according to the running parameters, wherein the sub-task execution command includes a task ID, a scene type and running parameters.

2. The information gathering method in penetration testing of claim 1, wherein, generating running parameters of the function module according to the scene type bound to each sub-intention data and the related constraint conditions, comprising: dynamically generating the number of threads, the scanning depth and the timeout parameters of the function module according to the scene type bound to each sub-intention data and the related constraint conditions.

3. The information gathering method in penetration testing of claim 1, wherein, Further comprising: identifying the dependency relationship between at least one sub-task, and generating a sub-task execution command corresponding to the sub-task according to the dependency relationship in the order of the dependency relationship for the sub-task with the dependency relationship, wherein the sub-task execution command includes the dependency relationship.

4. The information gathering method in penetration testing of any one of claims 1 to 3, wherein, The scene type includes: comprehensive collection scene, partial collection scene, specific target collection scene, fast collection scene, passive collection scene, continuous collection scene, target association collection scene and industry collection scene.

5. The information gathering method in penetration testing of any one of claims 1 to 3, wherein, The task scheduling strategy includes: breadth-first strategy, which is used to preferentially allocate execution among a plurality of different sub-tasks; depth-first strategy, which is used to preferentially execute along a sub-task chain; stealth-first strategy, which is used to preferentially select passive, low-interactive or low-risk sub-tasks for execution; risk-first strategy, which is used to preferentially schedule risk collection and detection tasks; and speed-first strategy, which is used to preferentially execute tasks with shorter time consumption or quickly obtainable results.

6. The information gathering method in penetration testing of any one of claims 1 to 3, wherein, dynamically selecting a task scheduling strategy suitable for each sub-task execution command according to the scene type included in each sub-task execution command, comprising: If the scene type is a fast collection scene, a speed priority strategy is selected; if the scene type is a comprehensive collection scene, a breadth priority strategy is selected; if the scene type is a passive collection scene, a concealment priority strategy is selected; if the scene type is a partial collection and specific target collection scene, a depth priority strategy is selected; if the scene type is a target association collection scene, a depth priority strategy is selected; and if the scene type is an industry collection scene, a risk priority strategy is selected.

7. An information gathering device in penetration testing, characterized by, Comprise: An intent recognition module, configured to receive an information collection instruction in a natural language form, identify a scene type of a penetration testing information collection scene corresponding to the information collection instruction based on a large language model, and decompose the information collection instruction into a plurality of sub-intent data based on the large language model, and bind each sub-intent data to the scene type corresponding to the information collection instruction; A command generation module, configured to dynamically generate a sub-task execution command corresponding to each sub-intent data according to each sub-intent data and the bound scene type; An information collection module, configured to dynamically select a task scheduling strategy suitable for each sub-task execution command according to the scene type included in each sub-task execution command, and call a corresponding function module to complete information collection based on the selected task scheduling strategy according to the sub-task execution command. The command generation module is configured to perform semantic recognition and relation extraction on each sub-intent data, obtain at least one sub-task, a task target, and related constraint conditions, determine a corresponding function module according to the scene type bound to each sub-intent data, the type of the sub-task, and the task target, generate a running parameter of the function module according to the scene type bound to each sub-intent data and the related constraint conditions, and generate the sub-task execution command according to the running parameter, wherein the sub-task execution command includes a task ID, a scene type, and a running parameter.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the information collection method in the penetration test in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program for executing the information collection method in the penetration test in any one of claims 1 to 6.

Images