An automated penetration testing method and system based on a cognitive decision model

Abstract

The application discloses an automatic penetration testing method and system based on a cognitive decision model, relates to the technical field of information security and data processing, and realizes deep cognitive modeling of a penetration environment by constructing a multilayer state space composed of an explicit state layer, a structural state layer and a potential cognitive state layer, directly records observable attributes through the explicit state layer, infers network topology and component correlation through graph analysis through the structural state layer, and speculates unknown factors based on a probability model through the potential cognitive state layer to form conditional probability representation, so that the modeling mode enables the system to construct complete environmental cognition from fragmented information, maintains decision stability through probabilistic reasoning when the information is incomplete, meanwhile, penetration experience trajectories are parsed into unified state-action sequences through a semantic mapping function from natural language penetration records, so that the trajectories ensure that expert reasoning logic is reflected, and a test system with environmental cognition and experience internalization is formed.

Description

Technical Field

[0001] This invention relates to the field of information security and data processing technology, specifically to an automated penetration testing method and system based on a cognitive decision-making model. Background Technology

[0002] With the widespread adoption of cloud-native, microservices, and zero-trust architectures, modern network systems have evolved into complex entities composed of massive dynamic components, whose attack surfaces are expanding exponentially and constantly changing. Traditional penetration testing relies heavily on the personal experience and manual reasoning of security experts, resulting in inherent bottlenecks such as inefficiency, incomplete coverage, and unreproducible results when facing large-scale, highly dynamic target environments. To address this challenge, automated penetration testing technology has become an inevitable direction for industry development, with its core objective being to enable machines to autonomously conduct security assessments and vulnerability chain discovery of complex systems.

[0003] Current automated penetration testing techniques fall into three categories: methods based on predefined rules and scripts, methods based on vulnerability knowledge bases and attack graphs, and methods based on reinforcement learning or large language models.

[0004] Methods based on predefined rules and scripts have extremely poor flexibility and generalization ability, and cannot cope with unknown scenarios;

[0005] The method based on vulnerability knowledge bases and attack graphs is essentially a static "fingerprint-vulnerability" matching, which lacks environmental understanding and reasoning capabilities and is difficult to handle new types of combined attacks.

[0006] Methods based on reinforcement learning or large language models are limited by the vast state and action space of the penetration task, the extremely sparse delayed reward signals, and the differences between simulated and real environments. This leads to difficulties in training, policy convergence, and weak transferability. While large language models can understand natural language and generate commands, they lack explicit modeling of task states, stable long-term planning capabilities, and mechanisms for continuous learning and correction from action results. When constructing multi-step, chain-like attack paths, they are prone to logical breaks and short-sighted decision-making problems.

[0007] In summary, existing technologies lack the ability to model and reproduce cognitive processes, meaning they cannot construct a holistic understanding of the target system from fragmented information or conduct multi-step planning based on experience and reasoning. Therefore, the industry urgently needs a new intelligent penetration testing method that can internalize expert experience and possess environmental awareness and forward-looking planning capabilities. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an automated penetration testing method and system based on a cognitive decision-making model. This method enables deep cognitive modeling of the penetration environment by constructing a multi-layered state space consisting of an explicit state layer, a structural state layer, and a potential cognitive state layer. The explicit state layer directly records observable attributes, the structural state layer infers network topology and component relationships through graph analysis, and the potential cognitive state layer infers unknown factors based on probabilistic models, forming conditional probability representations. This modeling approach allows the system to construct a complete environmental understanding from fragmented information and maintain decision stability through probabilistic reasoning when information is incomplete. Simultaneously, the penetration experience trajectory is parsed into a unified state-action sequence from natural language penetration records through a semantic mapping function and transformed into a learnable vector using an encoding function, ensuring that the trajectory reflects expert reasoning logic. This forms a testing system of environmental cognition and experience internalization, enabling the agent to learn attack and defense strategies from historical experience and quickly adapt to new environments.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On one hand, an automated penetration testing method based on a cognitive decision-making model, the specific steps of which are as follows:

[0010] S100. Constructing a cognitive state space for penetration tasks. The cognitive state space It includes an explicit state layer, a structural state layer, and a potential cognitive state layer, based on the cognitive state space. This formalizes the penetration testing task into a cognitive Markov decision process.

[0011] S200. Extract the operational and reasoning processes from the natural language penetration records using natural language processing techniques, and transform them into a structured penetration experience trajectory. The penetration experience trajectory A learnable state-action-value sequence;

[0012] S300: Train a multi-network collaborative decision-making model, including a policy network, a value network, and a state inference network, so that the collaborative decision-making model has the ability to make behavioral choices, long-term value judgments, and state predictions.

[0013] S400. In actual penetration testing, starting from the current cognitive state, and combining the policy network, value network, and state inference network, a Monte Carlo tree search with enhanced inference is run to perform multi-step forward planning and select actions to be executed. After executing the selected actions in the real environment, the cognitive state space of S100 is dynamically updated according to environmental feedback to incrementally generate new penetration experience trajectories, and further triggers the continuous training of the collaborative decision-making model in S300, forming an intelligent penetration testing process with autonomous reasoning and dynamic cognitive updates.

[0014] Furthermore, in S100, the cognitive state space is used to uniformly represent the observable explicit states, structural states, and potential cognitive states in the target network environment, specifically including:

[0015] The explicit state layer is used to record the explicit states that are directly observable in the task scenario, including accessible interfaces, identified component types, and features directly revealed by the environment.

[0016] The structural state layer is based on explicit state cues and constructs dynamically corrected structural states, which include network topology, relationships between components, and lateral expansion paths, and continuously corrects the structural states as the environment changes.

[0017] The latent cognitive state layer predicts the latent cognitive states of components through probabilistic modeling, represented as conditional probabilities. , This is a potential cognitive state. This is an explicit state. The structure is in a certain state;

[0018] The overall state space is represented as And based on the cognitive Markov decision process, it is represented as ,in, For a set of actions, For state transition model, This is the reward function.

[0019] Furthermore, the potential cognitive state The probabilistic modeling satisfies That is, the potential cognitive state is determined by a combination of the observable information of the explicit state and the reasoning results of the structural state, based on the explicit state. The system records component types and features directly revealed by the environment, matches them against a pre-defined vulnerability component association rule base, and initially filters potential vulnerability types, combined with structural status. The network topology and component relationships are analyzed to calculate the probability of exploitable paths to potential vulnerabilities using Bayesian probability. Output the probability distribution of potential cognitive states, representing a quantitative prediction of components and relationships.

[0020] Furthermore, in S200, the process of automatically extracting structured experience trajectories from natural language penetration records is as follows:

[0021] Natural language processing technology is used to perform semantic recognition and correlation analysis on behavioral fragments, environmental changes and thought clues in natural language penetration records;

[0022] Using semantic mapping functions The identified semantic information is mapped to states and actions, where... For natural language text, For state, For the action, the state The actions used to describe the characteristics of a penetration scenario Used to indicate executable penetration actions;

[0023] Constructing standardized experience trajectories ,in, This is the reward value;

[0024] Use encoding functions Transforming states into learnable vectors allows empirical trajectories to be used for model training.

[0025] Furthermore, the reward value The calculation is based on a multi-level reward system, which includes: cognitive gain rewards for measuring the improvement in environmental understanding, attack surface proximity rewards for characterizing the degree of attack, and salience rewards for marking breakthrough behaviors, and negative rewards for ineffective and contradictory behaviors.

[0026] Furthermore, in S300, the specific process of training the collaborative decision-making model includes:

[0027] The policy network uses a policy loss function. Training is conducted to learn action selection preferences under different task conditions, among which, This represents the policy loss function, used to measure the difference between the current action selection probability distribution of the policy network and the actual actions in the penetration experience trajectory. For policy network parameters, Represents the time step At that time, from the cognitive state space The current state is represented. Representative in the infiltration experience trajectory In the middle, in the state The actual actions taken;

[0028] The value network uses a value regression loss function. Training was conducted, including... This represents the value loss function, used to measure the gap between the predicted value of a value network and its actual long-term returns. To accumulate rewards, learn to assess the long-term benefits of a state. Representative value network in parameters Below, regarding the state The long-term value forecast;

[0029] The state deduction network uses a state deduction loss function. Training was conducted, including... To predict the current state, so as to learn to predict the future state. Represents the time step At that time, from the cognitive state space The current state is represented. This represents the state extrapolation loss function, used to measure the difference between the future state predicted by the state extrapolation network and the actual future state;

[0030] Through overall optimization function Perform multi-network collaborative training, where For strategy loss Value loss and state deduction loss The weighting coefficients.

[0031] Furthermore, the policy network, value network, and state inference network support equivalent replacements, including using graph neural networks, Transformers, and recurrent neural networks to replace the original network structures, while maintaining the synergy of behavior selection, value assessment, and state prediction.

[0032] Furthermore, in S400, the specific process based on Monte Carlo tree search combined with the state deduction model includes:

[0033] Using the current state as the root node of the search tree, the action probability distribution is generated through the policy network to provide verification direction for the tree search;

[0034] When selecting nodes, the upper confidence boundary is used. A balance between exploration and utilization, among which, For node value estimation, For the number of times a node is accessed. This represents the total number of visits to the parent node. To explore the strength coefficient;

[0035] When expanding to new nodes, the state deduction network is used to predict the next state after the action is performed, an exploration branch is generated internally, and the reward of the simulated path is evaluated through the value network. and based on Update node value;

[0036] Ultimately, the action is selected based on the number of times the node is accessed. implement.

[0037] On the other hand, an automated penetration testing system based on a cognitive decision-making model includes:

[0038] The cognitive modeling module is used to construct and maintain a cognitive state space consisting of an explicit state layer, a structural state layer, and a potential cognitive state layer.

[0039] The experience extraction and structuring module is used to extract and generate structured penetration experience trajectories from natural language penetration records.

[0040] The decision model training module is used to store and train the collaborative decision model composed of the policy network, value network and state inference network.

[0041] The intelligent planning and execution module is used to integrate the inference-enhanced Monte Carlo tree search logic and is responsible for controlling the execution and state update of actions in the real network environment after multi-step look-ahead planning.

[0042] In this system, the cognitive modeling module, experience extraction and structuring module, decision model training module, and intelligent planning and execution module interact and link logically based on a unified cognitive state space and the penetrating experience trajectory.

[0043] Compared with existing technologies, this automated penetration testing method based on a cognitive decision-making model has the following advantages:

[0044] This invention achieves deep cognitive modeling of the penetration environment by constructing a multi-layered state space consisting of an explicit state layer, a structural state layer, and a latent cognitive state layer. The explicit state layer directly records observable attributes, the structural state layer infers network topology and component relationships through graph analysis, and the latent cognitive state layer infers unknown factors based on a probabilistic model, forming a conditional probability representation. This modeling approach enables the system to construct a complete environmental cognition from fragmented information and maintain decision stability through probabilistic reasoning when information is incomplete. At the same time, the penetration experience trajectory is parsed into a unified state-action sequence through a semantic mapping function and transformed into a learnable vector by combining an encoding function, ensuring that the trajectory reflects the expert reasoning logic. This forms a testing system for environmental cognition and experience internalization, enabling the agent to learn attack and defense strategies from historical experience and adapt quickly to new environments.

[0045] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0047] Figure 1 This is a flowchart illustrating the steps of an automated penetration testing method based on a cognitive decision-making model.

[0048] Figure 2 This is a block diagram of the modular components of an automated penetration testing system based on a cognitive decision-making model.

[0049] Figure 3 This is a flowchart illustrating the operation of an automated penetration testing method based on a cognitive decision-making model. Detailed Implementation

[0050] To better understand the above technical solutions, a detailed description of the solutions will be provided below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0051] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “an automated penetration testing method based on a cognitive decision-making model,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise; “multiple” generally includes at least two.

[0052] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0053] To address the shortcomings of existing technologies, this invention first describes the automated penetration testing scenarios involved. This invention is primarily applied to security assessment scenarios in modern complex network environments, particularly dynamic and distributed systems under cloud-native, microservice, and zero-trust architectures. In these scenarios, the number of system components is enormous, the attack surface is constantly changing, and vulnerability exploitation paths are concealed and complexly interconnected. Traditional penetration testing methods relying on human experience or static scripts struggle to achieve comprehensive, efficient, and adaptive security assessments. This invention aims to achieve intelligent penetration testing with environmental understanding, autonomous reasoning, and dynamic learning capabilities by constructing a cognitive state space, internalizing expert experience trajectories, training a multi-network collaborative decision-making model, and combining it with forward-looking planning.

[0054] The automated penetration testing method based on a cognitive decision-making model provided by this invention achieves deep modeling of the target environment by constructing a multi-layered cognitive state space. It uses structured experience trajectories extracted from historical records as learning samples to train a collaborative decision-making model composed of a policy network, a value network, and a state inference network. In actual testing, it uses inference-enhanced Monte Carlo tree search for multi-step planning and action selection, forming a testing system of perception, reasoning, decision-making, and learning, thereby achieving autonomous, accurate, and evolvable penetration testing of complex network systems.

[0055] Specifically, such as Figure 1 As shown, an automated penetration testing method based on a cognitive decision-making model is described. The steps of this method are as follows:

[0056] S100. Constructing a cognitive state space for penetration tasks. The cognitive state space It includes an explicit state layer, a structural state layer, and a potential cognitive state layer, based on the cognitive state space. This formalizes the penetration testing task into a cognitive Markov decision process.

[0057] S200. Extract the operational and reasoning processes from the natural language penetration records using natural language processing techniques, and transform them into a structured penetration experience trajectory. The penetration experience trajectory A learnable state-action-value sequence;

[0058] S300: Train a multi-network collaborative decision-making model, including a policy network, a value network, and a state inference network, so that the collaborative decision-making model has the ability to make behavioral choices, long-term value judgments, and state predictions.

[0059] S400. In actual penetration testing, starting from the current cognitive state, and combining the policy network, value network, and state inference network, a Monte Carlo tree search with enhanced inference is run to perform multi-step forward planning and select actions to be executed. After executing the selected actions in the real environment, the cognitive state space of S100 is dynamically updated according to environmental feedback to incrementally generate new penetration experience trajectories, and further triggers the continuous training of the collaborative decision-making model in S300, forming an intelligent penetration testing process with autonomous reasoning and dynamic cognitive updates.

[0060] In the specific implementation process, step S100 is used to construct the cognitive state space S of the penetration task. The cognitive state space S is a formal representation of the environment in this embodiment, organizing the various types of information obtained during the penetration test into three levels: explicit state layer, structural state layer, and potential cognitive state layer. Based on this state space, the entire penetration task is formalized as a cognitive Markov decision process, providing a mathematical model foundation for subsequent decision-making and learning.

[0061] In the specific implementation of this embodiment, the explicit state layer is responsible for recording the raw information directly observed from the target network environment, including the results of initial reconnaissance actions such as port scanning, service identification, banner capture, and directory enumeration.

[0062] The structural state layer is based on explicit states for reasoning and construction, used to characterize the relationships between components and the overall topology of the target environment. For example, by identifying information such as IP addresses, domain names, and service types, it can infer subnetting, load balancing structures, front-end and back-end service dependencies, and database access paths. This structural state layer is dynamically corrected and expanded as the exploration deepens. The structural state layer relies on a graph model, treating network entities as nodes and access relationships, data flows, and trust relationships as edges, thereby forming a continuously updated environmental cognition graph.

[0063] The potential cognitive state layer uses probabilistic modeling to predict system attributes and vulnerability states that cannot be directly observed but exist. For example, after observing that a web application uses a specific framework version, and combining this with a known vulnerability database for that version, it predicts the likelihood of SQL injection or remote code execution vulnerabilities; or based on network topology and access control configuration, it infers that a jump server may have access to the core area. This potential cognitive state layer is represented as a conditional probability distribution. This involves quantifying and assessing various potential risks based on the current explicit and structural states, allowing the system to make reasonable and robust decisions even with incomplete information. This is a potential cognitive state. This is an explicit state. This refers to the structural state.

[0064] The overall cognitive state space is represented as Cognitive Markov Decision Process Based on This Definition ,in, For all executable penetration actions, This is a state transition model that describes how the state changes after an action is performed. The reward function is used to evaluate the immediate and long-term benefits of an action. Based on the cognitive Markov decision process, penetration testing is transformed into a sequential decision problem that can be optimized and solved by machine learning algorithms.

[0065] Step S200 involves extracting and transforming penetration experience trajectories from natural language penetration records. This method is used to transform unstructured natural language text into structured experience trajectories that can be learned by the model. In practice, it first utilizes Natural Language Processing (NLP) techniques, such as named entity recognition, relation extraction, and semantic role labeling, to perform deep analysis of the text, identifying the environmental information (corresponding to states) described within it. ), the operation to be performed (corresponding action) ), and the results of the operation (corresponding rewards) ).

[0066] Through a predefined semantic mapping function The identified semantic information is mapped to a unified state-action representation framework, where the state... Encoded to cover

[0067] A three-level vector or graph structure, actions This is then mapped to a standardized action library. For the reasoning process implicit in the text, this semantic mapping function establishes states. With action The causal relationships between them construct standardized experience trajectories. In this embodiment, the reward value Adopt a multi-tiered reward system:

[0068] Cognitive gain reward: When an action significantly increases understanding of the target system (such as discovering a new host or identifying a new service), a positive reward is given;

[0069] Attack surface proximity reward: When an action brings the system closer to core assets or elevates privilege levels (such as gaining ordinary user privileges or accessing the internal network), a positive reward is given;

[0070] Significant Breakthrough Rewards: When an action directly leads to the achievement of a key objective (such as acquiring sensitive data or fully controlling the system), a high positive reward is given.

[0071] Meanwhile, negative rewards are set for behaviors such as invalid loops, triggering alarms, or causing access loss.

[0072] Each state in the trajectory Through an encoding function Transform into a low-dimensional dense vector representation This is so that the data can be input into the neural network for training.

[0073] Step S300 involves training a multi-network collaborative decision-making model and utilizing the empirical trajectories generated in S200. Train a collaborative model consisting of a policy network, a value network, and a state inference network to enable the system to possess the core capability of intelligent decision-making, wherein:

[0074] The policy network uses a policy loss function. The input for training is the current state. The vector representation of the vector, the output of which is the selection of each possible action in this state. The probability distribution is such that the training objective is to maximize the expected cumulative reward, which is optimized using the policy gradient method. This represents the policy loss function, used to measure the difference between the current action selection probability distribution of the policy network and the actual actions in the penetration experience trajectory. For policy network parameters, Represents the time step At that time, from the cognitive state space The current state is represented. Representative in the infiltration experience trajectory In the middle, in the state The actual actions taken.

[0075] The value network uses a value regression loss function. Evaluate a given state The long-term value, that is, from that state Starting from there, the expected total return that can be obtained by following the current strategy, of which, This represents the value loss function, used to measure the gap between the predicted value of a value network and its actual long-term returns. To accumulate rewards, learn to assess the long-term benefits of a state. Representative value network in parameters Below, regarding the state The long-term value prediction value of the value network provides a long-term perspective for decision-making and helps avoid short-sighted behavior.

[0076] The state deduction network uses a state deduction loss function. Predicting the execution of actions Afterwards, the environmental state will be from Transferred to The probability distribution of , where, To predict the current state, so as to learn to predict the future state. Represents the time step At that time, from the cognitive state space The current state is represented. This represents the state extrapolation loss function, used to measure the difference between the future state predicted by the state extrapolation network and the actual future state.

[0077] These three networks are optimized through a single global optimization function. Conduct collaborative training, in which For strategy loss Value loss and state deduction loss The weight coefficients are used to train the policy network to learn to select high-value actions, the value network to learn to accurately evaluate the state, and the state inference network to learn to accurately predict environmental changes. It should be noted that the specific architecture of the network (such as using a fully connected network, graph neural network, or Transformer) can be replaced with equivalents according to the characteristics of the task, while keeping its core functions unchanged.

[0078] The S400 described above performs dynamic testing and learning by running an enhanced Monte Carlo tree search in actual penetration testing. When facing a new target network, it constructs an initial cognitive state based on the information obtained from the initial reconnaissance, which serves as the root node of the Monte Carlo tree search. The search process is as follows:

[0079] Selection: Starting from the root node, recursively select child nodes until a leaf node is reached. The selection strategy uses the upper confidence bound (UCB) formula. The choice is to balance the selection of known high-value nodes (utilization) and nodes with few visits (exploration), where, For node value estimation, For the number of times a node is accessed. This represents the total number of visits to the parent node. To explore the strength coefficient.

[0080] Expansion and Inference: When a leaf node is reached, the policy network provides the probability distribution of candidate actions, selects one of the actions for expansion, calls the state inference network to predict the next state to be reached after executing the action, and creates a virtual branch inside the search tree for inference, reducing the number of interactions with the real environment and improving planning efficiency.

[0081] Simulation Evaluation: On the internal inference branch, continue to use the policy network and state inference network for rapid simulation until a certain depth or terminal state is reached, and then use the value network to evaluate the final reward of the inference path.

[0082] Backtracking Update: Backtrack the simulated rewards along the search path and update the visit count of all nodes along the path. and valuation The updated formula is as follows .

[0083] Decision-making and execution: After multiple rounds of searching, the action with the most visits is selected based on the number of visits to the child nodes corresponding to each action under the root node. As the final decision, it is implemented in a real-world environment.

[0084] After an action is executed, the system collects real-world environmental feedback (such as command execution results, returned data, connection status, etc.), uses this feedback to dynamically update the current cognitive state space, and triggers continuous fine-tuning training of the collaborative decision-making model. It learns from each real interaction, constantly adapts to new environments, corrects the prediction model, and optimizes decision-making strategies.

[0085] like Figure 2 As shown, the operation flow of an automated penetration testing method based on a cognitive decision-making model is as follows:

[0086] (1) Constructing a multi-layered cognitive state space

[0087] Collect directly observable information from the target network, such as open interfaces and known component types, to form an explicit state layer.

[0088] Analyze the network topology and the relationships between components, and build and continuously update the structural state layer.

[0089] Based on the information from the first two layers, possible vulnerabilities and hidden states are predicted through a probabilistic model, forming a potential cognitive state layer.

[0090] (2) Extracting experience from historical infiltration records

[0091] Use natural language processing technology to analyze penetration test reports or operation logs.

[0092] Identify the "action-state-result" sequence in the records and transform it into a structured experience trajectory.

[0093] Each state and action is assigned an encoding, which transforms it into a vector form that can be used for model training.

[0094] (3) Training a multi-network collaborative decision-making model

[0095] Train the policy network to learn to select the most appropriate infiltration action under different conditions.

[0096] Train a value network to evaluate the long-term benefits of the current state.

[0097] The state projection network is trained to predict the next state that may occur after a certain action is performed.

[0098] The three networks are trained in a coordinated manner so that they can comprehensively consider behavioral choices, value assessments, and state predictions when making decisions.

[0099] (4) Conduct multi-step planning and execution in actual testing.

[0100] Starting from the current cognitive state, initiate a Monte Carlo tree search to plan multiple actions.

[0101] During the search process, the policy network guides action selection, the state inference network simulates future states, and the value network evaluates path benefits.

[0102] Select the action with the most visits as the action to be executed in the real environment.

[0103] Update the state space based on the execution results and record them as new experience trajectories.

[0104] Continuously train the model using new experiences to form a testing system that enables execution, learning, and updating.

[0105] (5) Continuous learning and cognitive updating

[0106] After each penetration test, the structural information and potential insights in the state space are automatically updated.

[0107] The newly generated experience trajectories are used for incremental training of the model, improving the intelligence and adaptability of subsequent tests.

[0108] On the other hand, such as Figure 3 As shown, the method provided above by the present invention is applicable to an automated penetration testing system based on a cognitive decision-making model, the system comprising:

[0109] The cognitive modeling module is used to construct and maintain a cognitive state space consisting of an explicit state layer, a structural state layer, and a potential cognitive state layer.

[0110] The experience extraction and structuring module is used to extract and generate structured penetration experience trajectories from natural language penetration records.

[0111] The decision model training module is used to store and train the collaborative decision model composed of the policy network, value network and state inference network.

[0112] The intelligent planning and execution module is used to integrate the inference-enhanced Monte Carlo tree search logic and is responsible for controlling the execution and state update of actions in the real network environment after multi-step look-ahead planning.

[0113] In this system, the cognitive modeling module, experience extraction and structuring module, decision model training module, and intelligent planning and execution module interact and logically link based on a unified cognitive state space and a permeated experience trajectory.

[0114] In summary, this invention constructs an intelligent penetration testing method that possesses deep environmental awareness, internalizes expert experience, enables multi-step forward planning, and continuously learns from practical experience. It combines human reasoning ability with machine computing, search, and iterative optimization capabilities, effectively solving the core problems of traditional methods in dealing with complex and dynamic network environments, such as incomplete coverage, poor adaptability, and inability to reason autonomously. This provides a feasible technical path for next-generation automated security assessment.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An automated penetration testing method based on a cognitive decision-making model, characterized in that, The steps of this method are as follows: S100. Constructing a cognitive state space for penetration tasks. The cognitive state space It includes an explicit state layer, a structural state layer, and a potential cognitive state layer, based on the cognitive state space. This formalizes the penetration testing task into a cognitive Markov decision process. S200. Extract the operational and reasoning processes from the natural language penetration records using natural language processing techniques, and transform them into a structured penetration experience trajectory. The penetration experience trajectory A learnable state-action-value sequence; S300: Train a multi-network collaborative decision-making model, including a policy network, a value network, and a state inference network, so that the collaborative decision-making model has the ability to make behavioral choices, long-term value judgments, and state predictions. S400. In actual penetration testing, starting from the current cognitive state, and combining the policy network, value network, and state inference network, a Monte Carlo tree search with enhanced inference is run to perform multi-step forward planning and select actions to be executed. After executing the selected actions in the real environment, the cognitive state space of S100 is dynamically updated according to environmental feedback to incrementally generate new penetration experience trajectories, and further triggers the continuous training of the collaborative decision-making model in S300, forming an intelligent penetration testing process with autonomous reasoning and dynamic cognitive updates.

2. The automated penetration testing method based on a cognitive decision-making model according to claim 1, characterized in that, In step S100, the cognitive state space is used to uniformly represent the observable explicit states, structural states, and potential cognitive states in the target network environment, specifically including: The explicit state layer is used to record the explicit states that are directly observable in the task scenario, including accessible interfaces, identified component types, and features directly revealed by the environment. The structural state layer is based on explicit state cues and constructs dynamically corrected structural states, which include network topology, relationships between components, and lateral expansion paths, and continuously corrects the structural states as the environment changes. The latent cognitive state layer predicts the latent cognitive states of components through probabilistic modeling, represented as conditional probabilities. , This is a potential cognitive state. This is an explicit state. The structure is in a certain state; The overall state space is represented as And based on the cognitive Markov decision process, it is represented as ,in, For a set of actions, For state transition model, This is the reward function.

3. The automated penetration testing method based on a cognitive decision-making model according to claim 2, characterized in that, The potential cognitive state The probabilistic modeling satisfies That is, the potential cognitive state is determined by a combination of the observable information of the explicit state and the reasoning results of the structural state, based on the explicit state. The system records component types and features directly revealed by the environment, matches them against a pre-defined vulnerability component association rule base, and initially filters potential vulnerability types, combined with structural status. The network topology and component relationships are analyzed to calculate the probability of exploitable paths to potential vulnerabilities using Bayesian probability. Output the probability distribution of potential cognitive states, representing a quantitative prediction of components and relationships.

4. The automated penetration testing method based on a cognitive decision-making model according to claim 1, characterized in that, In S200, the process of automatically extracting structured experience trajectories from natural language penetration records is as follows: Natural language processing technology is used to perform semantic recognition and correlation analysis on behavioral fragments, environmental changes and thought clues in natural language penetration records; Using semantic mapping functions The identified semantic information is mapped to states and actions, where... For natural language text, For state, For the action, the state The actions used to describe the characteristics of a penetration scenario Used to indicate executable penetration actions; Constructing standardized experience trajectories ,in, This is the reward value; Use encoding functions Transforming states into learnable vectors allows empirical trajectories to be used for model training.

5. The automated penetration testing method based on a cognitive decision-making model according to claim 4, characterized in that, The reward value The calculation is based on a multi-level reward system, which includes: cognitive gain rewards for measuring the improvement in environmental understanding, attack surface proximity rewards for characterizing the degree of attack, and salience rewards for marking breakthrough behaviors, and negative rewards for ineffective and contradictory behaviors.

6. An automated penetration testing method based on a cognitive decision-making model according to claim 1 or 4, characterized in that, In S300, the specific process of training the collaborative decision-making model includes: The policy network uses a policy loss function. Training is conducted to learn action selection preferences under different task conditions, among which, Represents the policy loss function. For policy network parameters, Represents the time step At that time, from the cognitive state space The current state is represented. Representative in the infiltration experience trajectory In the middle, in the state The actual actions taken; The value network uses a value regression loss function. Training was conducted, including... Represents the value loss function, To accumulate rewards, learn to assess the long-term benefits of a state. Representative value network in parameters Below, regarding the state The long-term value forecast; The state deduction network uses a state deduction loss function. Training was conducted, including... To predict the state, Represents the time step At that time, from the cognitive state space The current state is represented. This represents the state deduction loss function; Through overall optimization function Perform multi-network collaborative training, where For strategy loss Value loss and state deduction loss The weighting coefficients.

7. The automated penetration testing method based on a cognitive decision-making model according to claim 6, characterized in that, The policy network, value network, and state inference network support equivalent replacements, including using graph neural networks, Transformers, and recurrent neural networks to replace the original network structures, while maintaining the synergy of behavior selection, value assessment, and state prediction.

8. The automated penetration testing method based on a cognitive decision-making model according to claim 1, characterized in that, In S400, the specific process based on Monte Carlo tree search combined with the state deduction model includes: Using the current state as the root node of the search tree, the action probability distribution is generated through the policy network to provide verification direction for the tree search; When selecting nodes, the upper confidence boundary is used. A balance between exploration and utilization, among which, For node value estimation, For the number of times a node is accessed. This represents the total number of visits to the parent node. To explore the strength coefficient; When expanding to new nodes, the state deduction network is used to predict the next state after the action is performed, an exploration branch is generated internally, and the reward of the simulated path is evaluated through the value network. and based on Update node value; Ultimately, the action is selected based on the number of times the node is accessed. implement.

9. An automated penetration testing system based on a cognitive decision-making model, applicable to the automated penetration testing method based on a cognitive decision-making model as described in any one of claims 1-8, characterized in that, The system includes: The cognitive modeling module is used to construct and maintain a cognitive state space consisting of an explicit state layer, a structural state layer, and a potential cognitive state layer. The experience extraction and structuring module is used to extract and generate structured penetration experience trajectories from natural language penetration records. The decision model training module is used to store and train the collaborative decision model composed of the policy network, value network and state inference network. The intelligent planning and execution module is used to integrate the inference-enhanced Monte Carlo tree search logic and is responsible for controlling the execution and state update of actions in the real network environment after multi-step look-ahead planning. In this system, the cognitive modeling module, experience extraction and structuring module, decision model training module, and intelligent planning and execution module interact and link logically based on a unified cognitive state space and the penetrating experience trajectory.

Images