Honey bait deployment method and device fusing self-gaming and reinforcement learning, computer equipment, storage medium and computer program product

Abstract

The invention relates to a self-gaming and reinforcement learning fused honey bait deployment method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: determining a to-be-analyzed network node in a to-be-analyzed network in response to a honey bait deployment request for the to-be-analyzed network; acquiring attack state information, defense state information and node attribute information corresponding to the network node to be analyzed; inputting the attack state information, the defense state information and the node attribute information into a trained defender agent to obtain a node risk value corresponding to the network node to be analyzed; determining a node risk level corresponding to the to-be-analyzed network node according to the node risk value; generating a honey bait deployment instruction corresponding to the to-be-analyzed network node according to the attack state information and the node risk level; and according to the honey bait deployment instruction, performing corresponding honey bait deployment processing on the to-be-analyzed network node. By adopting the method, the honey bait deployment accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a honeypot deployment method, apparatus, computer device, computer-readable storage medium, and computer program product that integrates self-play and reinforcement learning. Background Technology

[0002] Currently, with the increasing severity of cybersecurity threats, accurately deploying honeypots within network nodes is crucial to ensuring their security.

[0003] In traditional techniques, bait deployment typically relies on preset rules and experience-based judgments; however, as attack techniques become more complex, this approach lacks intelligent dynamic adjustment capabilities, resulting in low accuracy in bait deployment. Summary of the Invention

[0004] Therefore, it is necessary to provide a honey bait deployment method, apparatus, computer device, computer-readable storage medium, and computer program product that integrates self-play and reinforcement learning to improve the accuracy of honey bait deployment, thereby addressing the aforementioned technical problems.

[0005] Firstly, this application provides a honeypot deployment method that integrates self-game theory and reinforcement learning, including:

[0006] In response to a honeypot deployment request for the network to be analyzed, the network nodes to be analyzed in the network to be analyzed are identified;

[0007] Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed;

[0008] The attack status information, the defense status information, and the node attribute information are input into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent.

[0009] Based on the node risk value, the node risk level corresponding to the network node to be analyzed is determined;

[0010] Based on the attack status information and the node risk level, generate honey bait deployment instructions corresponding to the network node to be analyzed;

[0011] According to the honey bait deployment instructions, the corresponding honey bait deployment process is performed on the network nodes to be analyzed.

[0012] In one embodiment, the trained defender agent is trained in the following manner:

[0013] Identify the sample network nodes in the sample network;

[0014] Obtain the sample attack status information, sample defense status information, and sample node attribute information corresponding to the sample network node;

[0015] The sample attack status information, the sample defense status information, and the sample node attribute information are input into the defender agent to be trained to obtain the sample node risk value corresponding to the sample network node.

[0016] Based on the risk value of the sample node, the risk level of the sample network node is determined.

[0017] Based on the sample attack status information and the risk level of the sample node, generate sample honey bait deployment instructions corresponding to the sample network node;

[0018] According to the sample honey bait deployment instruction, the sample attack status information, the sample defense status information, and the sample node attribute information are adjusted to obtain the adjusted attack status information, the adjusted defense status information, and the adjusted node attribute information.

[0019] The adjusted attack state information, the adjusted defense state information, and the adjusted node attribute information are input into the trained attacker agent to obtain the node attack command corresponding to the sample network node.

[0020] Based on the sample honey bait deployment instructions and the node attack instructions, the reward value corresponding to the sample network node is determined;

[0021] Based on the reward value corresponding to the sample network node, iterative adversarial training is performed on the defender agent to be trained to obtain the trained defender agent.

[0022] In one embodiment, determining the reward value corresponding to the sample network node based on the sample honeybee deployment instruction and the node attack instruction includes:

[0023] According to the sample bait deployment instruction, the sample network node is subjected to the corresponding bait deployment process to obtain the processed sample network node;

[0024] According to the node attack instruction, a node attack is performed on the processed sample network node to obtain the attack and defense results corresponding to the processed sample network node.

[0025] Based on the attack and defense results, the reward value corresponding to the sample network node is determined.

[0026] In one embodiment, the step of performing iterative adversarial training on the defender agent to be trained based on the reward value corresponding to the sample network node to obtain the trained defender agent includes:

[0027] Obtain the reward value corresponding to the associated network node; the associated network node is used to represent other network nodes besides the sample network node among the key network nodes of the sample network.

[0028] The reward values ​​corresponding to the sample network nodes and the reward values ​​corresponding to the associated network nodes are fused to obtain the target reward value.

[0029] Based on the target reward value, iterative adversarial training is performed on the defender agent to be trained to obtain the trained defender agent.

[0030] In one embodiment, determining the network nodes to be analyzed in the network to be analyzed includes:

[0031] Extract the network topology data of the network to be analyzed;

[0032] The network topology data is input into the trained critical path prediction model to obtain the critical paths in the network to be analyzed.

[0033] Candidate network nodes in the critical path are identified, and the node degree, node betweenness, node proximity, and node clustering degree of the candidate network nodes are obtained.

[0034] The node degree, node betweenness, node proximity, and node clustering degree are input into the trained node importance prediction model to obtain the predicted importance of the candidate network nodes.

[0035] From the candidate network nodes, candidate network nodes with a predicted importance greater than a preset importance are selected as the network nodes to be analyzed in the network to be analyzed.

[0036] In one embodiment, determining the node risk level corresponding to the network node to be analyzed based on the node risk value includes:

[0037] Obtain the current application scenario information of the network to be analyzed;

[0038] From multiple node risk level prediction models, the node risk level prediction model corresponding to the current application scenario information is selected as the target node risk level prediction model corresponding to the network node to be analyzed.

[0039] The node risk value is input into the target node risk level prediction model to obtain the node risk level corresponding to the network node to be analyzed.

[0040] Secondly, this application also provides a honeypot deployment device that integrates self-game theory and reinforcement learning, comprising:

[0041] The node determination module is used to determine the network nodes to be analyzed in the network to be analyzed in response to a honeypot deployment request for the network to be analyzed.

[0042] The information acquisition module is used to acquire the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed;

[0043] The risk value prediction module is used to input the attack status information, the defense status information, and the node attribute information into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent.

[0044] The risk level determination module is used to determine the node risk level corresponding to the network node to be analyzed based on the node risk value.

[0045] The instruction generation module is used to generate honey bait deployment instructions corresponding to the network node to be analyzed based on the attack status information and the node risk level.

[0046] The bait deployment module is used to perform corresponding bait deployment processing on the network node to be analyzed according to the bait deployment instructions.

[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0048] In response to a honeypot deployment request for the network to be analyzed, the network nodes to be analyzed in the network to be analyzed are identified;

[0049] Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed;

[0050] The attack status information, the defense status information, and the node attribute information are input into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent.

[0051] Based on the node risk value, the node risk level corresponding to the network node to be analyzed is determined;

[0052] Based on the attack status information and the node risk level, generate honey bait deployment instructions corresponding to the network node to be analyzed;

[0053] According to the honey bait deployment instructions, the corresponding honey bait deployment process is performed on the network nodes to be analyzed.

[0054] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0055] In response to a honeypot deployment request for the network to be analyzed, the network nodes to be analyzed in the network to be analyzed are identified;

[0056] Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed;

[0057] The attack status information, the defense status information, and the node attribute information are input into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent.

[0058] Based on the node risk value, the node risk level corresponding to the network node to be analyzed is determined;

[0059] Based on the attack status information and the node risk level, generate honey bait deployment instructions corresponding to the network node to be analyzed;

[0060] According to the honey bait deployment instructions, the corresponding honey bait deployment process is performed on the network nodes to be analyzed.

[0061] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0062] In response to a honeypot deployment request for the network to be analyzed, the network nodes to be analyzed in the network to be analyzed are identified;

[0063] Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed;

[0064] The attack status information, the defense status information, and the node attribute information are input into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent.

[0065] Based on the node risk value, the node risk level corresponding to the network node to be analyzed is determined;

[0066] Based on the attack status information and the node risk level, generate honey bait deployment instructions corresponding to the network node to be analyzed;

[0067] According to the honey bait deployment instructions, the corresponding honey bait deployment process is performed on the network nodes to be analyzed.

[0068] The aforementioned honeypot deployment method, apparatus, computer equipment, storage medium, and computer program product, which integrates game theory and reinforcement learning, first responds to a honeypot deployment request for the network to be analyzed by identifying the nodes in the network to be analyzed. Then, it acquires the attack status information, defense status information, and node attribute information corresponding to the nodes. Next, it inputs the attack status information, defense status information, and node attribute information into a trained defender agent to obtain the node risk value corresponding to the node to be analyzed. Based on the node risk value, it determines the node risk level corresponding to the node to be analyzed. Then, based on the attack status information and node risk level, it generates a honeypot deployment instruction for the node to be analyzed. Finally, it performs the corresponding honeypot deployment processing on the node to be analyzed according to the honeypot deployment instruction. The trained defender agent is obtained through iterative adversarial training using sample information corresponding to sample network nodes in the sample network and a trained attacker agent. In this way, when deploying honey bait, the core nodes of the network to be analyzed are the deployment targets. First, three types of objective data are comprehensively collected: attack status, defense status, and node attributes. Then, the defender's agent, which has been iteratively trained by the attacker's agent, accurately quantifies the node's risk value and classifies the risk level. Finally, the adaptive deployment instructions are generated by combining real-time attack characteristics and risk levels. This avoids the shortcomings of traditional technologies that rely on preset rules and experience-based judgments, which lack intelligent dynamic adjustment capabilities and result in low accuracy of honey bait deployment. It achieves precise adaptation of honey bait deployment to node risk and attack scenarios, greatly improving the targeting and effectiveness of honey bait deployment and thus improving the accuracy of honey bait deployment. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart illustrating a honey bait deployment method that integrates self-game theory and reinforcement learning in one embodiment.

[0071] Figure 2 This is a flowchart illustrating a honey bait deployment method that integrates self-game theory and reinforcement learning in another embodiment.

[0072] Figure 3 This is a schematic diagram of the intranet network structure in one embodiment;

[0073] Figure 4 This is a schematic diagram of the adversarial process in one embodiment;

[0074] Figure 5 This is a structural block diagram of a honey bait deployment device that integrates self-game theory and reinforcement learning in one embodiment.

[0075] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0077] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0078] In one exemplary embodiment, such as Figure 1 As shown, a honeypot deployment method integrating self-game theory and reinforcement learning is provided. This embodiment illustrates the method by applying it to a server; it is understood that the method can also be applied to terminals, and to systems including terminals and servers, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablets; the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. In this embodiment, the method includes the following steps:

[0079] Step S101: In response to the honeybee deployment request for the network to be analyzed, identify the network nodes to be analyzed in the network to be analyzed.

[0080] The network to be analyzed refers to the target network for which honeypot deployment is required, such as enterprise intranets, industrial control system networks, and cloud computing networks.

[0081] Among them, the honey bait deployment request refers to the request to deploy honey bait on the network to be analyzed.

[0082] Among them, the network nodes to be analyzed refer to the specific execution targets of the honey bait deployment.

[0083] For example, the server receives a honeybee deployment request sent by the terminal for the network to be analyzed, performs an integrity check on the honeybee deployment request, and obtains an integrity check result for the honeybee deployment request; if the integrity check result indicates that the honeybee deployment request has passed the check, the server responds to the honeybee deployment request and identifies the network nodes to be analyzed in the network to be analyzed.

[0084] Step S102: Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed.

[0085] The attack status information includes the attack behavior characteristics of the network node (such as the number of times the network node to be analyzed was attacked, the distribution of the attack sources, and the types of tools attacked), the attack intensity level (the degree to which the network node to be analyzed was attacked), and the attack intent tendency information (the attacker's target tendency corresponding to the network node to be analyzed, such as data theft, system damage, and lateral penetration).

[0086] The defense status information includes the defense measures configuration information of the network node (such as the type of security protection components deployed, the update time of protection rules, and the history of honey bait deployment for the network node to be analyzed) and the defense strength level (the degree to which the network node to be analyzed defends itself).

[0087] The node attribute information includes basic node attribute information (including hardware configuration, operating system type and version, and physical location of deployment of the network node to be analyzed), core business attribute information (including business type, business weight, and service running status of the network node to be analyzed), and topology association attribute information (including path start point and destination of the network node to be analyzed).

[0088] For example, the server collects initial attack status information, initial defense status information, and initial node attribute information corresponding to the network node to be analyzed through multiple types of built-in sensors (including attack behavior capture sensors, defense status monitoring sensors, and node attribute acquisition sensors) deployed on the network node to be analyzed; then, the initial attack status information, initial defense status information, and initial node attribute information are denoised to obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed.

[0089] Step S103: Input the attack status information, defense status information, and node attribute information into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; The trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network node in the sample network and the trained attacker agent.

[0090] Among them, the defender agent refers to a network model that can use attack status information, defense status information and node attribute information to obtain the honey bait deployment instructions corresponding to the network node to be analyzed, such as the near-end strategy optimization model.

[0091] Among them, the node risk value is used to represent the probability that the network node to be analyzed will be successfully attacked and the extent of the damage caused by the attack.

[0092] Among them, the sample network refers to the network used to train the defender agent.

[0093] Here, a sample network node refers to a network node in the sample network.

[0094] The sample information includes attack status information, defense status information, and node attribute information corresponding to the sample network nodes.

[0095] Among them, the attacker agent refers to a network model that can use attack status information, defense status information and node attribute information to obtain the node attack instructions corresponding to the network node to be analyzed, such as the near-end strategy optimization model.

[0096] Iterative adversarial training refers to a model training method that enables the defender agent to learn more robust deployment strategies through continuous adversarial training between two agents (i.e., the defender agent and the attacker agent).

[0097] For example, the server performs feature extraction processing on attack state information, defense state information, and node attribute information respectively to obtain a first feature vector of attack state information, a second feature vector of defense state information, and a third feature vector of node attribute information. The first, second, and third feature vectors are then concatenated to obtain a concatenated feature vector. Next, the concatenated feature vector is input into a trained defender agent to obtain a first node risk value corresponding to the network node to be analyzed. The concatenated feature vector is also input into a trained node risk value prediction model to obtain a second node risk value corresponding to the network node to be analyzed. Then, based on the model weights of the trained defender agent and the trained node risk value prediction model, the first and second node risk values ​​are summed to obtain the node risk value corresponding to the network node to be analyzed. The trained defender agent is obtained through iterative adversarial training using sample information corresponding to sample network nodes in the sample network and a trained attacker agent.

[0098] Step S104: Determine the node risk level corresponding to the network node to be analyzed based on the node risk value.

[0099] Among them, the node risk level is used to represent the degree of risk corresponding to the network node to be analyzed.

[0100] For example, the server determines the current node risk level of the network node to be analyzed based on the node risk value; then, it obtains the historical node risk level of the network node to be analyzed, and performs a fusion process on the current node risk level and the historical node risk level to obtain the node risk level of the network node to be analyzed.

[0101] Step S105: Based on the attack status information and node risk level, generate honey bait deployment instructions corresponding to the network nodes to be analyzed.

[0102] Among them, the honey bait deployment command refers to the executable command for deploying honey bait on the network nodes to be analyzed.

[0103] For example, when the node risk level is greater than the preset risk level, the server inputs the attack status information into the trained honey bait type prediction model to obtain the predicted probability of the network node to be analyzed under each preset honey bait type, and selects the preset honey bait type with the highest predicted probability from each preset honey bait type as the target honey bait type corresponding to the network node to be analyzed; then, according to the target honey bait type, the honey bait deployment instruction corresponding to the network node to be analyzed is generated.

[0104] Step S106: Deploy the corresponding honey bait on the network node to be analyzed according to the honey bait deployment instructions.

[0105] For example, the server performs a validity check on the honey bait deployment command and obtains the validity check result of the honey bait deployment command; if the honey bait deployment command indicates that the honey bait deployment command has passed the verification, the server performs the corresponding honey bait deployment processing on the network node to be analyzed in accordance with the honey bait deployment command.

[0106] In the aforementioned honeypot deployment method integrating game theory and reinforcement learning, the process first responds to honeypot deployment requests for the network to be analyzed by identifying the nodes in the network to be analyzed. Then, it acquires the attack status information, defense status information, and node attribute information corresponding to these nodes. Next, it inputs these information into a trained defender agent to obtain the node risk value. Based on this risk value, it determines the node risk level. Then, based on the attack status information and node risk level, it generates a honeypot deployment instruction for the node. Finally, it performs the corresponding honeypot deployment process on the node according to the instruction. The trained defender agent is obtained through iterative adversarial training using sample information from sample network nodes in the sample network and a trained attacker agent. In this way, when deploying honey bait, the core nodes of the network to be analyzed are the deployment targets. First, three types of objective data are comprehensively collected: attack status, defense status, and node attributes. Then, the defender's agent, which has been iteratively trained by the attacker's agent, accurately quantifies the node's risk value and classifies the risk level. Finally, the adaptive deployment instructions are generated by combining real-time attack characteristics and risk levels. This avoids the shortcomings of traditional technologies that rely on preset rules and experience-based judgments, which lack intelligent dynamic adjustment capabilities and result in low accuracy of honey bait deployment. It achieves precise adaptation of honey bait deployment to node risk and attack scenarios, greatly improving the targeting and effectiveness of honey bait deployment and thus improving the accuracy of honey bait deployment.

[0107] In an exemplary embodiment, the honey bait deployment method integrating self-game theory and reinforcement learning provided in this application further includes a training process for a trained defender agent, specifically including the following: identifying sample network nodes in the sample network; obtaining sample attack state information, sample defense state information, and sample node attribute information corresponding to the sample network nodes; inputting the sample attack state information, sample defense state information, and sample node attribute information into the defender agent to be trained to obtain the sample node risk value corresponding to the sample network node; determining the sample node risk level corresponding to the sample network node based on the sample node risk value; and generating a sample network based on the sample attack state information and the sample node risk level. The system generates sample bait deployment instructions for each node. Based on these instructions, it adjusts the sample attack state, defense state, and node attribute information to obtain adjusted attack state, defense state, and node attribute information. This adjusted information is then input into the trained attacker agent to obtain the node attack instructions for each sample network node. The system determines the reward value for each sample network node based on the sample bait deployment instructions and the node attack instructions. Finally, it performs iterative adversarial training on the defender agent to be trained based on the reward value for each sample network node, resulting in a trained defender agent.

[0108] Among them, sample attack status information refers to the attack status information corresponding to the sample network nodes.

[0109] Among them, sample defense status information refers to the defense status information corresponding to the sample network nodes.

[0110] Among them, sample node attribute information refers to the node attribute information corresponding to the sample network node.

[0111] Among them, the sample node risk value refers to the node risk value corresponding to the sample network node.

[0112] Among them, the risk level of a sample node refers to the risk level of the node corresponding to the sample network node.

[0113] Among them, the sample honey bait deployment instruction refers to the honey bait deployment instruction corresponding to the sample network node.

[0114] Among them, the adjusted attack status information refers to the sample attack status information after being adjusted based on the sample bait deployment instructions.

[0115] Among them, the adjusted defense status information refers to the sample defense status information after being adjusted based on the sample honey bait deployment instructions.

[0116] The adjusted node attribute information refers to the sample node attribute information after being adjusted based on the sample bait deployment instructions.

[0117] Among them, node attack instructions refer to structured instructions used to simulate attacking sample network nodes.

[0118] The reward value refers to the quantitative feedback value calculated based on the adversarial results (attack and defense results) between the sample bait deployment command and the node attack command.

[0119] For example, in response to the agent training instructions for the defender agent to be trained, the server identifies sample network nodes in the sample network; then, it retrieves sample attack state information, sample defense state information, and sample node attribute information corresponding to the sample network nodes from the database; next, it inputs the sample attack state information, sample defense state information, and sample node attribute information into the defender agent to be trained to obtain the sample node risk value corresponding to the sample network node; then, based on the sample node risk value, it determines the sample node risk level corresponding to the sample network node; then, based on the sample attack state information and the sample node risk level, it generates a sample honey bait deployment instruction corresponding to the sample network node; next, it extracts the instruction parameters from the sample honey bait deployment instruction and adjusts them based on attack state adjustment rules (such as "after database honey bait deployment, the attack type tilts towards SQL injection") and defense state adjustment rules (such as "after adding honey bait, the defense strength quantification value..."). The attack status information, defense status information, and node attribute information of the sample are adjusted according to the rules for improving the score by 1-2 points and the attribute information adjustment rules (such as adding the 'honey bait node' association tag to the honey bait deployment path). The adjusted attack status information, defense status information, and node attribute information are obtained. Then, the adjusted attack status information, defense status information, and node attribute information are input into the trained attacker agent to obtain the predicted probability of the sample network node under each preset node attack command. The preset node attack command with the highest predicted probability is selected from the preset node attack commands and used as the node attack command corresponding to the sample network node. Next, the reward value corresponding to the sample network node is determined according to the sample honey bait deployment command and the node attack command. Then, the defender agent to be trained is iteratively trained according to the reward value corresponding to the sample network node to obtain the trained defender agent.

[0120] It should be noted that the reward value corresponding to the sample network node is also used to adjust the model parameters of the trained attacker agent, resulting in an attacker agent with adjusted model parameters, which is then applied to the training process of the defender agent in the next round to achieve a self-game effect.

[0121] In this embodiment, by constructing an iterative adversarial training closed loop with two agents, the training effectiveness is ensured by maintaining consistency between the sample data and the structure and attack and defense characteristics of the real network. Furthermore, by leveraging the attacker agent's dynamic attack response to the bait deployment command, the defender agent learns accurate risk assessment capabilities and bait deployment strategies to adapt to complex attack scenarios, which is beneficial to improving the quality of model training.

[0122] In an exemplary embodiment, the reward value corresponding to the sample network node is determined according to the sample bait deployment instruction and the node attack instruction. Specifically, this includes: deploying the sample network node according to the sample bait deployment instruction to obtain the processed sample network node; attacking the processed sample network node according to the node attack instruction to obtain the attack and defense result corresponding to the processed sample network node; and determining the reward value corresponding to the sample network node based on the attack and defense result.

[0123] Among them, the attack and defense result refers to the quantitative result of the simulated attack launched on the processed sample network nodes based on the node attack command.

[0124] For example, the server performs honeypot deployment processing on the sample network nodes according to the sample honeypot deployment instructions to obtain the processed sample network nodes; then, according to the node attack instructions, it performs node attacks on the processed sample network nodes to obtain the attack capture status quantification value (including capture success rate and attack chain integrity) and defense interception effect quantification value (including attack interception status and defense breakthrough degree) corresponding to the processed sample network nodes, which are used as the attack and defense results corresponding to the processed sample network nodes; then, according to the weight coefficients corresponding to the attack capture status quantification value and the defense interception effect quantification value, the attack capture status quantification value and the defense interception effect quantification value are summed to obtain the reward value corresponding to the sample network nodes.

[0125] In this embodiment, by deploying honey bait, simulating attacks in a scenario-based manner, and quantifying the attack and defense results, the consistency between the honey bait deployment instructions and the sample honey bait deployment instructions, as well as the fit between the simulated attack and the real attack logic, are ensured. This allows the reward value to objectively reflect the effectiveness of the sample honey bait deployment strategy and the quality of the defender agent's decision-making, which is beneficial to improving the robustness of the model.

[0126] In an exemplary embodiment, the defender agent to be trained is subjected to iterative adversarial training based on the reward value corresponding to the sample network node to obtain the trained defender agent. Specifically, this includes: obtaining the reward value corresponding to the associated network node; fusing the reward value corresponding to the sample network node and the reward value corresponding to the associated network node to obtain the target reward value; and performing iterative adversarial training on the defender agent to be trained based on the target reward value to obtain the trained defender agent.

[0127] Among them, the associated network nodes are used to represent the other network nodes besides the sample network nodes in the key network nodes of the sample network.

[0128] Among them, critical network nodes refer to network nodes whose importance is greater than the preset importance.

[0129] The target reward value refers to the comprehensive reward value obtained by fusing the reward values ​​corresponding to the sample network nodes and the reward values ​​corresponding to the associated network nodes.

[0130] For example, the server obtains the reward value corresponding to the associated network node; based on the importance of the sample network node and the associated network node, it determines the weight coefficient of the reward value corresponding to the sample network node and the weight coefficient of the reward value corresponding to the associated network node; then, according to the weight coefficient of the reward value corresponding to the sample network node and the weight coefficient of the reward value corresponding to the associated network node, it sums the reward values ​​corresponding to the sample network node and the associated network node to obtain the target reward value; then, based on the target reward value, it adjusts the model parameters of the defender agent to be trained to obtain the defender agent with adjusted model parameters, and based on the target reward value, it adjusts the model parameters of the attacker agent that has been trained to obtain the attacker agent with adjusted model parameters; then, it uses the defender agent with adjusted model parameters as the defender agent to be trained, and the attacker agent with adjusted model parameters as the attacker agent that has been trained, and jumps to the step of determining the sample network node in the sample network, until the iteration round of the defender agent with adjusted model parameters reaches the preset iteration round, and then uses the defender agent with adjusted model parameters as the defender agent that has been trained.

[0131] In this embodiment, by introducing the reward value of associated network nodes, the evaluation of the defense effect of a single node is extended to the evaluation of the global collaborative defense effect. Through fusion processing, the limitations of isolated node evaluation in traditional training are broken, providing more comprehensive and objective feedback information for the defender agent to be trained, and enabling more accurate adaptation to the complex scenarios of node interconnection in real networks.

[0132] In an exemplary embodiment, step S101, which identifies the network nodes to be analyzed in the network to be analyzed, specifically includes the following: extracting the network topology data of the network to be analyzed; inputting the network topology data into a trained critical path prediction model to obtain the critical paths in the network to be analyzed; identifying candidate network nodes in the critical paths and obtaining the node degree, node betweenness, node proximity, and node clustering degree of the candidate network nodes; inputting the node degree, node betweenness, node proximity, and node clustering degree into a trained node importance prediction model to obtain the predicted importance of the candidate network nodes; and selecting candidate network nodes whose predicted importance is greater than a preset importance from among the candidate network nodes as the network nodes to be analyzed in the network to be analyzed.

[0133] Network topology data refers to a set of structured data used to describe the connection relationships and structural characteristics between nodes and links in the network to be analyzed.

[0134] Among them, the critical path prediction model refers to the network model used to predict the critical path in the network to be analyzed, such as the convolutional neural network model.

[0135] The critical path refers to the path with the highest importance in the network to be analyzed.

[0136] Candidate network nodes refer to all network nodes included in the critical path.

[0137] Among them, node degree refers to the number of other nodes directly connected to a candidate network node (divided into in-degree: the number of connections pointing to the node; and out-degree: the number of connections from the node to other nodes).

[0138] Among them, the node betweenness number refers to the proportion of the number of shortest paths in the network to be analyzed that pass through the candidate network nodes.

[0139] Node proximity refers to the reciprocal of the average shortest path length from a candidate network node to other network nodes in the network being analyzed.

[0140] Among them, node clustering degree refers to the degree of interconnection between the direct neighbors of a candidate network node (such as the ratio of the actual number of connected pairs among the neighbors to the theoretical maximum number of connected pairs).

[0141] Among them, the node importance prediction model refers to the network model used to obtain the predicted importance of candidate network nodes, such as the recurrent neural network model.

[0142] Among them, predicted importance refers to the predicted value of the importance of candidate network nodes.

[0143] The preset importance level refers to a pre-defined threshold for importance. It should be noted that the preset importance level depends on the specific circumstances.

[0144] For example, the server inputs the network to be analyzed into a trained information extraction model, which extracts the network topology data of the network to be analyzed. Then, key network topology data is extracted from the network topology data and input into a trained critical path prediction model to obtain the importance of each path in the network to be analyzed. From each path, the path with the highest importance is determined as the critical path in the network to be analyzed. Next, candidate network nodes in the critical path are determined, and their node degree, betweenness, proximity, and clustering degree are calculated. Then, feature vectors corresponding to the node degree, betweenness, proximity, and clustering degree are constructed, and these feature vectors are input into a trained node importance prediction model to obtain the predicted importance of the candidate network nodes. Finally, from all candidate network nodes, those with predicted importance greater than a preset importance are selected as the network nodes to be analyzed in the network to be analyzed.

[0145] In this embodiment, the core paths that attackers can easily exploit are accurately located by using a trained critical path prediction model, avoiding the blindness of traditional indiscriminate analysis of the entire network. Furthermore, by integrating multi-dimensional topological features such as node degree and betweenness, the importance of candidate nodes is quantitatively evaluated through a node importance prediction model, overcoming the one-sidedness of single feature evaluation and providing an accurate foundation for subsequent data analysis.

[0146] In an exemplary embodiment, step S104 above, which determines the node risk level of the network node to be analyzed based on the node risk value, specifically includes the following: obtaining the current application scenario information of the network to be analyzed; selecting the node risk level prediction model corresponding to the current application scenario information from multiple node risk level prediction models, and using it as the target node risk level prediction model for the network node to be analyzed; inputting the node risk value into the target node risk level prediction model to obtain the node risk level of the network node to be analyzed.

[0147] The current application scenario information refers to the specific business domain in which the network to be analyzed is located, such as financial transactions and industrial control.

[0148] Among them, the node risk level prediction model refers to the network model used to predict the risk level of a network node, including gradient boosting trees, neural network models, etc. It should be noted that each node risk level prediction model corresponds to an application scenario.

[0149] Among them, the target node risk level prediction model refers to the node risk level prediction model that matches the network nodes to be analyzed.

[0150] For example, the server automatically extracts the current application scenario information of the network to be analyzed from the network configuration file of the network to be analyzed; then, based on the current application scenario information, it queries the correspondence between the application scenario information and the model type to obtain the target model type corresponding to the current application scenario information; then, from multiple node risk level prediction models, it selects the node risk level prediction model corresponding to the target model type as the target node risk level prediction model for the node to be analyzed; then, it inputs the node risk value into the target node risk level prediction model to obtain the prediction probability of the node to be analyzed under each preset node risk level, and selects the preset node risk level with the highest prediction probability from each preset node risk level as the node risk level corresponding to the node to be analyzed.

[0151] In this embodiment, by accurately acquiring the current application scenario information of the network to be analyzed, the limitation of a single model adapting to all scenarios in traditional node risk level prediction is broken. This enables the selected target node risk level prediction model to deeply fit the business characteristics, data sensitivity and security requirements of the current scenario, which is conducive to improving the accuracy of determining the node risk level.

[0152] In one exemplary embodiment, such as Figure 2 As shown, another honey bait deployment method integrating self-game theory and reinforcement learning is provided. Taking the application of this method to a server as an example, the specific steps include:

[0153] Step S201: In response to the honeypot deployment request for the network to be analyzed, extract the network topology data of the network to be analyzed.

[0154] Step S202: Input the network topology data into the trained critical path prediction model to obtain the critical path in the network to be analyzed.

[0155] Step S203: Identify candidate network nodes in the critical path and obtain the node degree, node betweenness, node proximity, and node clustering degree of the candidate network nodes.

[0156] Step S204: Input the node degree, node betweenness, node proximity, and node clustering degree into the trained node importance prediction model to obtain the predicted importance of candidate network nodes.

[0157] Step S205: Select candidate network nodes with predicted importance greater than preset importance from each candidate network node, and use them as network nodes to be analyzed in the network to be analyzed.

[0158] Step S206: Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed.

[0159] Step S207: Input the attack status information, defense status information, and node attribute information into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; The trained defender agent is obtained by iterative adversarial training on the defender agent to be trained using the sample information corresponding to the sample network node in the sample network and the trained attacker agent.

[0160] Step S208: Obtain the current application scenario information of the network to be analyzed.

[0161] Step S209: Select the node risk level prediction model that corresponds to the current application scenario information from multiple node risk level prediction models, and use it as the target node risk level prediction model for the network node to be analyzed.

[0162] Step S210: Input the node risk value into the target node risk level prediction model to obtain the node risk level corresponding to the network node to be analyzed.

[0163] Step S211: Based on the attack status information and node risk level, generate honey bait deployment instructions corresponding to the network nodes to be analyzed.

[0164] Step S212: According to the honey bait deployment instructions, perform corresponding honey bait deployment processing on the network nodes to be analyzed.

[0165] The aforementioned honey bait deployment method, which integrates game theory and reinforcement learning, first comprehensively collects three types of objective data—attack status, defense status, and node attributes—by targeting the core nodes of the network to be analyzed during honey bait deployment. Then, a defender agent, trained iteratively against the attacker agent, accurately quantifies the node risk value and classifies the risk level. Finally, it generates adaptive deployment instructions by combining real-time attack characteristics and risk levels. This avoids the shortcomings of traditional techniques that rely on preset rules and experience-based judgments, which lack intelligent dynamic adjustment capabilities and result in low accuracy of honey bait deployment. It achieves precise adaptation of honey bait deployment to node risk and attack scenarios, significantly improving the targeting and effectiveness of honey bait deployment and thus enhancing its accuracy.

[0166] In one exemplary embodiment, to more clearly illustrate the honeypot deployment method integrating self-game theory and reinforcement learning provided in this application, the following specific embodiment will be used to describe the honeypot deployment method integrating self-game theory and reinforcement learning. In one embodiment, this application also provides yet another honeypot deployment method integrating self-game theory and reinforcement learning. Specifically, it includes the following:

[0167] 1. Model building:

[0168] In communication and computing infrastructure networks, attackers aim to penetrate critical nodes to gain access to key components, while defenders deploy honeypots to detect potential attack strategies. A typical network topology is as follows: Figure 3 As shown.

[0169] The above network structure can be abstractly represented as a graph. In this network, node N represents a component, such as the attacker's computer in a penetration attack scenario, the target component the attacker wants to compromise, and other components in the network. Edge E represents the connection relationship between components.

[0170] Each node Each has a corresponding node state. .in, This indicates the attack status of a node, containing m attributes representing the strength of different types of attacks against that node, and is only visible to the attacker. Indicates the defense status of a node, including Each attribute represents the defense strength against m types of attacks and the node's attack detection capability; these attributes are only visible to the defender. All attribute values ​​range from 0 to... Within the range of natural numbers. The size of the entire Markov game's state space is... It covers all possible combinations of node states.

[0171] Attackers on visible nodes Two actions can be performed. The first is a reconnaissance action, which allows the attacker to obtain the node's defense status. Information; secondly, attack actions, targeting one of m attack types, which can increase the corresponding attack value. Increase by 1. If the attack value exceeds the node's defense value. If the attack succeeds, the node is compromised, and its neighboring nodes become visible to the attacker. If the attack fails, the attack will... The probability of being detected by the defender depends on the node's detection capability. Decision. The defender at the node. There are also two action options. Monitoring actions can improve the node's detection capabilities and increase the detection value. Increase; defensive actions are used to enhance defense against specific types of attacks, increasing the corresponding defense value. Increase. The action space size for both attackers and defenders is [increased / increased]. It takes into account both the number of nodes and the types of offensive and defensive actions.

[0172] The attack and defense process is conducted in a turn-based manner. In each turn, the attacker and defender each perform one action. The system transitions from the current state to the next state based on the actions of both sides. This transition process exhibits the Markov property, i.e. It depends only on the current state and current action, and is independent of historical states. The attack and defense ends when the attacker successfully compromises the target node. (Attacker wins) or the attacker is detected by the defender (defender wins). The winning side receives a utility reward of +1, while the losing side receives a utility reward of -1. This reward mechanism incentivizes both sides to optimize their respective strategies.

[0173] 2. Algorithm Implementation:

[0174] In this game environment constructed using a graph model, a self-playing approach combined with reinforcement learning is employed to learn strategies. Initially, random strategies are assigned to both the attacker and the defender. Subsequently, they engage in multiple rounds of adversarial simulations based on their current strategies. After each round of simulation, the strategies are updated using the game results and trajectories, leveraging the reinforcement learning algorithm. This process is repeated until the strategies of both sides converge, resulting in an effective security strategy.

[0175] In this scheme, honey bait deployment is used as an important defensive tactic. In each round of combat, the defender can choose to deploy honey bait at a specific node. After deployment, the vulnerability value (det) of that node increases by 1, and the detection attribute value... Similarly, by increasing the value by 1, this method induces attackers to discover a large vulnerability value after detecting the node's state, thus prompting them to attack. The higher detection value... It also makes it easier for the defender to detect the attacker's attack behavior.

[0176] Because the state space of graph models has high dimensionality, growing exponentially with the number of nodes and attack types, traditional tabular reinforcement learning methods are difficult to apply. Therefore, deep neural networks are used to achieve function approximation, representing the policy as a parameterized function. The network takes the observation value 'o' as input and outputs the conditional probability distribution of all possible actions. Simultaneously estimate the value function This effectively solves the problems caused by high-dimensional state spaces.

[0177] The specific solution is as follows:

[0178] (1) Deep Neural Networks (DNN): Using the Actor-Critic architecture parameterization strategy Input observation o (such as node state), output action probability distribution.

[0179] (2) Value function estimation: The Critic network simultaneously outputs the state value. Used to calculate the advantage function This reduces the variance of gradient estimation.

[0180] To address the issues of large action space and slow learning convergence in graph models, an autoregressive policy representation is proposed. Actions are decomposed into two sub-actions: node selection and attack / defense type selection. The policy is then represented according to the probability chain rule. Decomposed into and Implemented using neural networks, the action space size is increased from... Reduce to This accelerates strategy convergence.

[0181] The specific solution is as follows:

[0182] (1) Action decomposition: Decompose the action into a two-step decision:

[0183] Node Selection: Strategy Target node n is selected based on observation.

[0184] Action Type Selection: Strategy Select the action type (such as attack type or hardening type) on node n.

[0185] (2) Probability chain rule: , from the action space Dimensional reduction .

[0186] In self-games, the environment becomes non-stationary due to simultaneous learning by both sides, making strategy convergence difficult. To address this, an opponent pool technique is introduced. During training, the current offensive and defensive strategies are periodically added to the pool, and opponent strategies are sampled from the pool during training. This increases strategy diversity, reduces overfitting, and improves the likelihood of strategy convergence in self-games.

[0187] The specific solution is as follows:

[0188] (1) Historical strategy pool: During the training process, the current strategy is saved to the pool periodically to form a diverse set of opponent strategies.

[0189] (2) Random sampling mechanism: During training, the opponent's strategy is randomly selected from the pool to avoid overfitting of a single strategy and improve the robustness of the strategy.

[0190] In summary, the confrontation process is as follows: Figure 4 As shown.

[0191] In training mode, the agent interacts with the environment in multiple rounds. The agent updates its policy by calling its train method to improve the cumulative reward gained in the environment.

[0192] By configuring training parameters, creating an attack and defense environment, and initializing two independent agents (attacker and defender), the agents are trained through multiple rounds of training. The greedy strategy selects actions and interacts with the environment, stores experience in the replay buffer, periodically samples data from the buffer to update the Q network to optimize the strategy, and periodically updates the target network to ensure training stability. As training progresses, the exploration rate is gradually decayed to make the agent turn to policy exploitation, and the agent's performance is evaluated at regular intervals. Ultimately, the attacker and defender optimize and improve their strategies in the adversarial environment.

[0193] (1) Initialization phase: The program first configures the training parameters, creates the attack and defense environment, and initializes two independent agents (attacker and defender). Each agent has its own policy network and experience replay buffer.

[0194] (2) Training main loop: The system executes multiple training rounds (specified by num_episodes), each round containing the following key steps:

[0195] Reset the environment status and initialize the cumulative reward counter.

[0196] Interaction Phase: While the environment has not reached its termination state, the two agents continue to interact.

[0197] Step 1, based on the current state and Greedy strategy, where the attacker and defender each choose their own actions.

[0198] Step 2: The environment receives the actions from both parties, calculates the reward, and moves to the next state.

[0199] Step 3: Store the experience (state, action, reward, next state, termination flag) into their respective experience replay buffers.

[0200] Step 4: Update cumulative rewards.

[0201] Step 5: Sample data from the replay buffer and update the agent's Q network (perform backpropagation optimization).

[0202] Step 6, Regular Updates: After each target_update_freq round, update the target network to stabilize training.

[0203] Step 7, Exploration Rate Decay: Gradually Reduce This value allows intelligent agents to shift from exploration to utilization.

[0204] Step 8, Performance Evaluation: Evaluate and output the agent's average reward every eval_frequency rounds.

[0205] (3) Core mechanism:

[0206] Experience replay: Improve sample utilization efficiency and training stability by storing and randomly sampling historical experience.

[0207] Target network: Periodically synchronize the parameters of the target network to reduce fluctuations in Q-value estimation during training.

[0208] Greedy strategy: Balances exploration and exploitation, gradually focusing on the optimal strategy as training progresses.

[0209] In the above embodiments, when deploying honey bait, the core nodes of the network to be analyzed are the deployment targets. First, three types of objective data—attack status, defense status, and node attributes—are comprehensively collected. Then, a defender agent, iteratively trained against the attacker agent, accurately quantifies the node risk value and classifies the risk level. Finally, adaptive deployment instructions are generated by combining real-time attack characteristics and risk levels. This avoids the shortcomings of traditional technologies that rely on preset rules and experience-based judgments, which lack intelligent dynamic adjustment capabilities and result in low accuracy in honey bait deployment. It achieves precise adaptation of honey bait deployment to node risk and attack scenarios, significantly improving the targeting and effectiveness of honey bait deployment and thus increasing its accuracy. Simultaneously, this method exhibits excellent adaptability and stability in complex network environments. It can dynamically adjust defense strategies based on attacker behavior, effectively responding to constantly changing network threats. Through multi-round game learning, the system gradually optimizes the honey bait deployment strategy, significantly improving the detection and defense capabilities against unknown attacks. The designed intelligent decision-making model, by defining reasonable state space, action space, and reward function, achieves autonomous learning and optimization of the honey bait deployment strategy. The model effectively captures the dynamic characteristics of the network environment and continuously adjusts its decision-making strategies based on the game process between attackers and defenders. This adaptive learning mechanism enables the defense system to continuously improve its adversarial capabilities and effectively respond to new attack methods. The constructed network adversarial model successfully simulates the game process between attackers and defenders, and represents the attack and defense states numerically, providing a reliable basis for evaluating the effectiveness of defense strategies. Experimental results show that the model can accurately reflect the attack and defense adversarial situation in the real network environment, providing an important reference for optimizing defense strategies. Overall, the honeypot deployment method and system that integrates self-game theory and reinforcement learning provides an innovative defense approach for the field of network security, laying the foundation for building a more intelligent and efficient network security defense system. This method has significant advantages in improving network security defense capabilities.

[0210] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0211] Based on the same inventive concept, this application also provides a honeypot deployment device for implementing the honeypot deployment method that integrates self-game theory and reinforcement learning as described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the honeypot deployment device integrating self-game theory and reinforcement learning provided below can be found in the limitations of the honeypot deployment method integrating self-game theory and reinforcement learning described above, and will not be repeated here.

[0212] In one exemplary embodiment, such as Figure 5 As shown, a honey bait deployment device integrating self-game theory and reinforcement learning is provided, comprising: a node determination module 501, an information acquisition module 502, a risk value prediction module 503, a level determination module 504, an instruction generation module 505, and a honey bait deployment module 506, wherein:

[0213] The node determination module 501 is used to determine the network nodes to be analyzed in the network in response to a honeypot deployment request for the network to be analyzed.

[0214] The information acquisition module 502 is used to acquire the attack status information, defense status information and node attribute information of the network node to be analyzed.

[0215] The risk value prediction module 503 is used to input attack status information, defense status information and node attribute information into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained through sample information corresponding to sample network nodes in the sample network and the trained attacker agent.

[0216] The risk level determination module 504 is used to determine the risk level of the network node to be analyzed based on the node risk value.

[0217] The instruction generation module 505 is used to generate honey bait deployment instructions for the network nodes to be analyzed based on the attack status information and node risk level.

[0218] The honey bait deployment module 506 is used to perform corresponding honey bait deployment processing on the network nodes to be analyzed according to the honey bait deployment instructions.

[0219] In an exemplary embodiment, the honey bait deployment device integrating self-game theory and reinforcement learning further includes a model training module, used to determine sample network nodes in the sample network; acquire sample attack state information, sample defense state information, and sample node attribute information corresponding to the sample network nodes; input the sample attack state information, sample defense state information, and sample node attribute information into the defender agent to be trained to obtain the sample node risk value corresponding to the sample network node; determine the sample node risk level corresponding to the sample network node based on the sample node risk value; and generate sample honey bait deployment corresponding to the sample network node based on the sample attack state information and the sample node risk level. The instructions are as follows: Based on the sample bait deployment instructions, adjust the sample attack state information, sample defense state information, and sample node attribute information to obtain adjusted attack state information, adjusted defense state information, and adjusted node attribute information; input the adjusted attack state information, adjusted defense state information, and adjusted node attribute information into the trained attacker agent to obtain the node attack instructions corresponding to the sample network nodes; determine the reward value corresponding to the sample network nodes based on the sample bait deployment instructions and node attack instructions; perform iterative adversarial training on the defender agent to be trained based on the reward value corresponding to the sample network nodes to obtain the trained defender agent.

[0220] In an exemplary embodiment, the model training module is further configured to perform corresponding honey bait deployment processing on the sample network nodes according to the sample honey bait deployment instruction, to obtain the processed sample network nodes; perform node attack on the processed sample network nodes according to the node attack instruction, to obtain the attack and defense results corresponding to the processed sample network nodes; and determine the reward value corresponding to the sample network nodes based on the attack and defense results.

[0221] In an exemplary embodiment, the model training module is further configured to obtain the reward value corresponding to the associated network node; the associated network node is used to represent other network nodes besides the sample network node among the key network nodes of the sample network; the reward value corresponding to the sample network node and the reward value corresponding to the associated network node are fused to obtain the target reward value; based on the target reward value, the defender agent to be trained is subjected to iterative adversarial training to obtain the trained defender agent.

[0222] In an exemplary embodiment, the node determination module 501 is further configured to: extract network topology data of the network to be analyzed; input the network topology data into a trained critical path prediction model to obtain critical paths in the network to be analyzed; determine candidate network nodes in the critical paths and obtain the node degree, node betweenness, node proximity, and node clustering degree of the candidate network nodes; input the node degree, node betweenness, node proximity, and node clustering degree into a trained node importance prediction model to obtain the predicted importance of the candidate network nodes; and select candidate network nodes whose predicted importance is greater than a preset importance from among the candidate network nodes as the network nodes to be analyzed in the network to be analyzed.

[0223] In an exemplary embodiment, the level determination module 504 is further configured to obtain the current application scenario information of the network to be analyzed; select the node risk level prediction model corresponding to the current application scenario information from multiple node risk level prediction models, and use it as the target node risk level prediction model corresponding to the node of the network to be analyzed; input the node risk value into the target node risk level prediction model to obtain the node risk level corresponding to the node of the network to be analyzed.

[0224] The modules in the honeypot deployment device integrating game theory and reinforcement learning described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0225] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores attack status information, defense status information, and other data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a honeypot deployment method that integrates self-game theory and reinforcement learning.

[0226] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0227] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0228] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0229] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0230] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0231] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0232] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A honey bait deployment method integrating self-game theory and reinforcement learning, characterized in that, The method includes: In response to a honeypot deployment request for the network to be analyzed, the network nodes to be analyzed in the network to be analyzed are identified; Obtain the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed; The attack status information, the defense status information, and the node attribute information are input into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent. Based on the node risk value, the node risk level corresponding to the network node to be analyzed is determined; Based on the attack status information and the node risk level, generate honey bait deployment instructions corresponding to the network node to be analyzed; According to the honey bait deployment instructions, the corresponding honey bait deployment process is performed on the network nodes to be analyzed.

2. The method according to claim 1, characterized in that, The trained defender agent was obtained through the following method: Identify the sample network nodes in the sample network; Obtain the sample attack status information, sample defense status information, and sample node attribute information corresponding to the sample network node; The sample attack status information, the sample defense status information, and the sample node attribute information are input into the defender agent to be trained to obtain the sample node risk value corresponding to the sample network node. Based on the risk value of the sample node, the risk level of the sample network node is determined. Based on the sample attack status information and the risk level of the sample node, generate sample honey bait deployment instructions corresponding to the sample network node; According to the sample honey bait deployment instruction, the sample attack status information, the sample defense status information, and the sample node attribute information are adjusted to obtain the adjusted attack status information, the adjusted defense status information, and the adjusted node attribute information. The adjusted attack state information, the adjusted defense state information, and the adjusted node attribute information are input into the trained attacker agent to obtain the node attack command corresponding to the sample network node. Based on the sample honey bait deployment instructions and the node attack instructions, the reward value corresponding to the sample network node is determined; Based on the reward value corresponding to the sample network node, iterative adversarial training is performed on the defender agent to be trained to obtain the trained defender agent.

3. The method according to claim 2, characterized in that, The step of determining the reward value corresponding to the sample network node based on the sample honey bait deployment instruction and the node attack instruction includes: According to the sample bait deployment instruction, the sample network node is subjected to the corresponding bait deployment process to obtain the processed sample network node; According to the node attack instruction, a node attack is performed on the processed sample network node to obtain the attack and defense results corresponding to the processed sample network node. Based on the attack and defense results, the reward value corresponding to the sample network node is determined.

4. The method according to claim 2, characterized in that, The step of iteratively adversarially training the defender agent to be trained based on the reward value corresponding to the sample network node to obtain the trained defender agent includes: Obtain the reward value corresponding to the associated network node; the associated network node is used to represent other network nodes besides the sample network node among the key network nodes of the sample network. The reward values ​​corresponding to the sample network nodes and the reward values ​​corresponding to the associated network nodes are fused to obtain the target reward value. Based on the target reward value, iterative adversarial training is performed on the defender agent to be trained to obtain the trained defender agent.

5. The method according to claim 1, characterized in that, The process of determining the network nodes to be analyzed in the network to be analyzed includes: Extract the network topology data of the network to be analyzed; The network topology data is input into the trained critical path prediction model to obtain the critical paths in the network to be analyzed. Candidate network nodes in the critical path are identified, and the node degree, node betweenness, node proximity, and node clustering degree of the candidate network nodes are obtained. The node degree, node betweenness, node proximity, and node clustering degree are input into the trained node importance prediction model to obtain the predicted importance of the candidate network nodes. From the candidate network nodes, candidate network nodes with a predicted importance greater than a preset importance are selected as the network nodes to be analyzed in the network to be analyzed.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the node risk level corresponding to the network node to be analyzed based on the node risk value includes: Obtain the current application scenario information of the network to be analyzed; From multiple node risk level prediction models, the node risk level prediction model corresponding to the current application scenario information is selected as the target node risk level prediction model corresponding to the network node to be analyzed. The node risk value is input into the target node risk level prediction model to obtain the node risk level corresponding to the network node to be analyzed.

7. A honey bait deployment device integrating self-game theory and reinforcement learning, characterized in that, The device includes: The node determination module is used to determine the network nodes to be analyzed in the network to be analyzed in response to a honeypot deployment request for the network to be analyzed. The information acquisition module is used to acquire the attack status information, defense status information, and node attribute information corresponding to the network node to be analyzed; The risk value prediction module is used to input the attack status information, the defense status information, and the node attribute information into the trained defender agent to obtain the node risk value corresponding to the network node to be analyzed; the trained defender agent is obtained by iterative adversarial training of the defender agent to be trained using the sample information corresponding to the sample network nodes in the sample network and the trained attacker agent. The risk level determination module is used to determine the node risk level corresponding to the network node to be analyzed based on the node risk value. The instruction generation module is used to generate honey bait deployment instructions corresponding to the network node to be analyzed based on the attack status information and the node risk level. The bait deployment module is used to perform corresponding bait deployment processing on the network node to be analyzed according to the bait deployment instructions.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images