Network defense method, device, apparatus and storage medium

Abstract

This invention relates to the field of network security technology and discloses a network defense method, apparatus, device, and storage medium. The method is applied to a security defense system deployed in a network environment, where device nodes and honeypot nodes are deployed, and the device nodes deploy defense strategies. The method includes: acquiring reward information from the network environment based on the defense strategies, where the reward information represents the current attack losses after deploying the defense strategies; receiving attacker information from each honeypot node after inducing an attacker to launch an attack; and determining whether to redeploy the defense strategies based on the reward information and the attacker information. This invention combines the reward information from the network environment and the attacker information from the honeypot nodes to comprehensively determine whether to redeploy the defense strategies. Compared to the static defense strategies of existing technologies, the above method of this invention can flexibly respond to changing attack methods and effectively improve security defense efficiency.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a network defense method, apparatus, device, and storage medium. Background Technology

[0002] Currently, existing network defense methods involve firewalls taking a series of countermeasures to protect network and system security when they detect potential threats or violations of security policies. Firewalls can blacklist source IP addresses with malicious intent or known attack behavior based on configured rules, thereby blocking their access to the network.

[0003] However, the countermeasures mentioned above usually require manual intervention or configuration to respond, which are static defense strategies that cannot flexibly cope with constantly changing attack methods, resulting in low security defense efficiency.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a network defense method, apparatus, device, and storage medium, which aims to solve the technical problem that existing countermeasures usually require manual intervention or configuration to respond, are static defense strategies, and cannot flexibly cope with constantly changing attack methods, resulting in low security defense efficiency.

[0006] To achieve the above objectives, the present invention provides a network defense method, which is applied to a security defense system deployed in a network environment. The network environment has device nodes and honeypot nodes deployed, both of which are targets of attackers. The honeypot nodes are nodes that induce attackers to launch attacks, and the device nodes have defense strategies deployed in them.

[0007] The network defense methods include:

[0008] Obtain reward information from the network environment based on the defense strategy, wherein the reward information is the attack loss information currently suffered after the deployment of the defense strategy;

[0009] Receive attacker information fed back by each honeypot node after it has induced the attacker to attack;

[0010] Based on the reward information and the attacker information, determine whether to redeploy the defense strategy.

[0011] Optionally, the step of determining whether to redeploy the defense strategy based on the reward information and the attacker information includes:

[0012] Determine the historical estimate of the device node, which is a measure of the effectiveness of the device node's defense against attacks under the defense strategy;

[0013] Determine whether the current moment is within a preset time step that supports the redeployment of the defense strategy;

[0014] If so, the historical estimate is updated based on the reward information to obtain the updated estimate;

[0015] The exploration entropy value of the device node is determined by the number of times the defense strategy has been deployed in the past.

[0016] Based on the updated estimate, the exploration entropy value, and the attacker information, a decision is made as to whether to redeploy the defense strategy.

[0017] Optionally, the step of determining whether to redeploy the defense strategy based on the updated estimate, the exploration entropy value, and the attacker information includes:

[0018] Based on the attacker information, determine the abnormal information metric corresponding to the device node;

[0019] The current estimate of the device node is determined by the confidence interval upper bound algorithm based on the updated estimate, the exploration entropy value, and the anomaly information metric, and the estimates of other device nodes in the network environment are also determined.

[0020] Compare the current estimate with the estimates of the other device nodes;

[0021] When the current estimate is the maximum estimate, it is determined that there is no need to redeploy the defense strategy, which is a strategy for controlling the device node to perform defense actions.

[0022] Optionally, after the step of determining that the defense strategy does not need to be redeployed when the current estimate is the maximum estimate, the method further includes:

[0023] Determine the number of currently available services in the network environment;

[0024] The network availability metric is determined based on the number of available services in the current environment and the number of available services in the initial environment.

[0025] Determine whether the network availability metric has reached a preset metric threshold;

[0026] If so, return to the step of obtaining the reward information fed back by the network environment based on the defense strategy.

[0027] Optionally, after the step of comparing the current estimate with the estimates of the other device nodes, the method further includes:

[0028] When the current estimate is not the maximum estimate, the target device node with the maximum estimate is selected from the other device nodes;

[0029] Determine the attribute information of the target device node, and determine whether the target device node supports the deployment of the defense strategy based on the attribute information;

[0030] If supported, the defense strategy is deployed to the target device node so that the target device node performs the defense action based on the defense strategy.

[0031] Optionally, before the step of receiving attacker information returned by each honeypot node after inducing the attacker to attack, the method further includes:

[0032] Extract the environmental feature vector from the network environment;

[0033] The expected reward value of different honeypot nodes is calculated based on the environmental feature vector using the linear confidence interval upper bound algorithm. The expected reward value represents the probability of each honeypot node being selected.

[0034] Based on the expected reward value, the desired honeypot node is selected from the different honeypot nodes.

[0035] Optionally, the method further includes:

[0036] When the honeypot node needs to be switched, determine the attack damage information between the deployment time of the honeypot node and the current time;

[0037] Update the environmental feature vector based on the attack damage information, and return to the step of calculating the expected reward value of different honeypot nodes based on the environmental feature vector using the linear confidence interval upper bound algorithm.

[0038] Furthermore, to achieve the above objectives, the present invention also proposes a network defense device, the device comprising:

[0039] The reward acquisition module is used to acquire reward information fed back by the network environment based on the defense strategy. The reward information is the attack loss information currently suffered after the deployment of the defense strategy.

[0040] The honeypot feedback module is used to receive attacker information returned by each honeypot node after it has lured the attacker to attack.

[0041] The strategy deployment module is used to determine whether to redeploy the defense strategy based on the reward information and the attacker information.

[0042] Furthermore, to achieve the above objectives, the present invention also proposes a network defense device, the device comprising: a memory, a processor, and a network defense program stored in the memory and executable on the processor, the network defense program being configured to implement the steps of the network defense method described above.

[0043] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a network defense program, which, when executed by a processor, implements the steps of the network defense method described above.

[0044] This invention provides a network defense method, apparatus, device, and storage medium. The method is applied to a security defense system deployed in a network environment, where device nodes and honeypot nodes are deployed. Both device nodes and honeypot nodes are targets of attackers, with honeypot nodes acting as bait to induce the attackers to launch attacks. The device nodes contain defense strategies. The method includes: acquiring reward information from the network environment based on the defense strategies, where the reward information represents the current attack losses after deploying the defense strategies; receiving attacker information from each honeypot node after inducing the attackers to launch attacks; and determining whether to redeploy the defense strategies based on the reward information and the attacker information. This invention comprehensively determines whether to redeploy the defense strategies by combining the reward information from the network environment and the attacker information from the honeypot nodes. Compared to the static defense strategies of existing technologies, the method of this invention allows for adjustments based on both reward information and attacker information after deployment, enabling flexible responses to constantly changing attack methods and effectively improving security defense efficiency. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the network defense device structure in the hardware operating environment involved in the embodiments of the present invention;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the network defense method of the present invention;

[0047] Figure 3 This is a flowchart illustrating the attack and defense interaction in the first embodiment of the network defense method of the present invention.

[0048] Figure 4 This is a flowchart illustrating the second embodiment of the network defense method of the present invention;

[0049] Figure 5 This is a flowchart of the defense algorithm in the second embodiment of the network defense method of the present invention;

[0050] Figure 6 This is a flowchart illustrating the third embodiment of the network defense method of the present invention;

[0051] Figure 7This is a flowchart of the honeypot proactive defense algorithm in the third embodiment of the network defense method of the present invention;

[0052] Figure 8 This is a structural block diagram of the first embodiment of the network defense device of the present invention.

[0053] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0055] Reference Figure 1 , Figure 1 This is a schematic diagram of the network defense device structure in the hardware operating environment involved in the embodiments of the present invention.

[0056] like Figure 1 As shown, the network defense device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0057] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on network defense equipment and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0058] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a network defense program.

[0059] exist Figure 1In the network defense device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the network defense device of the present invention can be set in the network defense device, and the network defense device calls the network defense program stored in the memory 1005 through the processor 1001 and executes the network defense method provided in the embodiment of the present invention.

[0060] This invention provides a network defense method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the network defense method of the present invention.

[0061] In this embodiment, the method is applied to a security defense system deployed in a network environment. The network environment includes device nodes and honeypot nodes, both of which are targets of attackers. The honeypot nodes are nodes used to lure attackers into launching attacks. The device nodes are equipped with defense strategies. The network defense method includes the following steps:

[0062] Step S10: Obtain the reward information fed back by the network environment based on the defense strategy, wherein the reward information is the attack loss information currently suffered after the deployment of the defense strategy.

[0063] It should be noted that the executing entity of the method in this embodiment can be a computing service device with network defense, network communication, and program execution functions, such as a mobile phone, tablet computer, or personal computer, or other electronic devices that perform the same or similar functions. The following description uses the aforementioned network defense system to illustrate this embodiment and the subsequent embodiments.

[0064] Understandably, the aforementioned device nodes can be hardware devices deployed in the network environment, such as computers or mobile phones.

[0065] It should be noted that the honeypot nodes mentioned above can be simulated vulnerable nodes based on network environment defense scenarios. Systems with honeypot nodes deployed can simulate real network services or systems, enticing attackers to attack them, thereby analyzing the attacker's attack intentions and methods, strengthening the system's defense capabilities, and improving the ability to identify and respond to real attacks.

[0066] Understandably, the aforementioned defense strategy can be a machine restart strategy. If a defense strategy is deployed on a device node, the device node will restart when the network environment is attacked from the outside, in order to quickly restore the normal operation of the system and enhance the system's security, thereby effectively defending against external attacks.

[0067] It should be noted that the reward information mentioned above represents the current attack loss information after the aforementioned defense strategy has been deployed. This attack loss information can refer to the losses or costs incurred by the network environment after being attacked by an attacker, such as lost data, damaged systems or services, etc.

[0068] In addition, the above reward information can be used to participate in adjusting the defense strategy. Specifically, if the total attack loss is low, it means that the defense strategy may be more effective, so a positive reward can be given. Conversely, if the total attack loss is high, it means that the defense strategy may not be effective enough and needs further learning and improvement.

[0069] In its implementation, the aforementioned network defense system can deploy the defense strategy on one or more device nodes within the network environment. When the network environment is attacked, the device node with the defense strategy deployed will be restarted to defend against external attacks. Since the device node with the defense strategy is not the attacker's target, the network defense system will still incur some losses after the network environment is attacked. After a certain number of attacks, the network environment will report the total attack loss information suffered by the system under the defense strategy and will feed this total attack loss information back to the network defense system as reward information, so that the network defense system is aware of the current losses.

[0070] Step S20: Receive attacker information fed back by each honeypot node after it has induced the attacker to attack.

[0071] It should be noted that the attacker information mentioned above refers to information about the attacker's current attack, including the attacker's attack methods, attack behaviors, and attack damage.

[0072] In practice, the attackers mentioned above not only target device nodes but also honeypot nodes. After a honeypot node is attacked, it can collect attacker information and report this information to the network defense system, enabling the system to understand the attacker's intentions and methods based on this information.

[0073] Step S30: Determine whether to redeploy the defense strategy based on the reward information and the attacker information.

[0074] In its implementation, the aforementioned network defense system can determine the total attack loss based on reward information, identify the attacker's intent and attack methods based on attacker information, and then analyze the defense effectiveness of the device nodes currently deployed with defense strategies by combining the total attack loss, the attacker's intent, and the attack methods. If the effectiveness is good, the defense strategy does not need to be redeployed, and the previous device nodes can still be restarted. If the effectiveness is poor, the defense strategy can be redeployed to other device nodes to switch to other device nodes for restart, thereby ensuring the defense effectiveness of the device nodes.

[0075] It should be understood that after redeploying the defense strategy, the operation of obtaining reward information from the network environment based on the defense strategy can be repeated, that is, the above process can be repeated to dynamically adjust the defense strategy to adapt to constantly changing attack methods.

[0076] It should be noted that traditional network defense technologies typically employ static defense strategies. These strategies capture network traffic through network interfaces or devices and transmit it to a network monitoring module for analysis. The network monitoring module then forwards the network traffic and behavior data to an attack detection and analysis module. This module uses predefined rules and algorithms to analyze the traffic, identify potential attack behaviors, and pass the relevant information to a firewall module to take appropriate protective measures. However, because attackers' methods are constantly evolving, static defense strategies cannot respond in a timely manner and often require manual intervention or configuration for prompt action, resulting in slow response times and low defense effectiveness. In contrast, the network defense system in this embodiment can dynamically adjust its defense strategy based on feedback from the network environment and attacker information, automatically selecting to restart or switch machines without manual intervention. This allows for better handling of constantly changing attack methods, effectively improving defense effectiveness and accuracy.

[0077] For ease of understanding, please refer to Figure 3 This explanation does not limit the scope of this solution. Figure 3 In this network defense system, a defender module and a honeypot proactive defense module can be included. The defender module can select device nodes to deploy defense strategies based on the defense effectiveness of each device node; that is, it selects the nodes in the network that need to perform defense actions (which may be restarts), and adjusts the defense strategy based on reward information provided by the network environment and anomaly information detected by the honeypot nodes. The honeypot proactive defense module can redeploy honeypot nodes based on environmental characteristics and feedback, enabling the honeypot nodes to collect more accurate anomaly information and improving the accuracy of the defender module's defense strategy adjustment.

[0078] The aforementioned environmental characteristics can be network-related features, such as the node's operating system type, the services it provides, and recent network traffic. The environmental feedback can also represent the damage suffered by the network defense system between two honeypot deployments.

[0079] This embodiment applies to a security defense system deployed in a network environment. The network environment includes device nodes and honeypot nodes, both of which are targets of attackers. The honeypot nodes are used to lure attackers into launching attacks, while the device nodes contain defense strategies. The method includes: acquiring reward information from the network environment based on the defense strategies, where the reward information represents the current attack losses after deploying the defense strategies; receiving attacker information from each honeypot node after it has lured an attacker into an attack; and determining whether to redeploy the defense strategies based on the reward information and the attacker information. This embodiment comprehensively determines whether to redeploy the defense strategies by combining the reward information from the network environment and the attacker information from the honeypot nodes. Compared to the static defense strategies of existing technologies, this embodiment allows for adjustments based on both reward information and attacker information after deployment, enabling flexible responses to constantly changing attack methods and effectively improving security defense efficiency.

[0080] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the network defense method of the present invention.

[0081] Based on the first embodiment described above, in this embodiment, step S30 includes:

[0082] Step S301: Determine the historical estimate of the device node, wherein the historical estimate is a measure of the degree of danger when the device node is attacked.

[0083] In its implementation, the aforementioned network defense system can determine the historical estimate of the number of times the device node with the deployed defense strategy was attacked before the last restart.

[0084] It should be understood that the above-mentioned risk level measurement can be preset according to different attacks. For example, the harm level of the attack behavior of discovering nodes is lower than the harm level of the attack behavior of controlling nodes.

[0085] Step S302: Determine whether the current moment is within a preset time step that supports the redeployment of the defense strategy.

[0086] It should be noted that the aforementioned preset time step can be the time period during which the attacker executes the Nth attack action, where N can be a predefined value. That is, when the attacker has executed N-1 attack actions and begins to execute the Nth action, it is determined that the current moment has entered the preset time step.

[0087] In practice, if the attacker adjusts the defense strategy without performing an attack action, it will lead to system instability. Therefore, the above-mentioned preset time step is defined so that the network defense system can only perform the subsequent redeployment of the defense strategy after entering the above-mentioned preset time step at the current moment, so as to ensure system stability.

[0088] Step S303: If so, update the historical estimate based on the reward information to obtain the updated estimate.

[0089] In its specific implementation, after the network defense system detects that the current time is at the preset time step, it can determine the degree of harm caused by the vulnerability exploited in the system from the last restart to the current time based on the reward information. Then, it adds the degree of harm measurement to the historical estimate to obtain the updated estimate.

[0090] Step S304: Determine the exploration entropy value of the device node by the number of times the defense strategy has been deployed in the past.

[0091] It should be noted that the exploration entropy value mentioned above represents the number of times the device node has been attempted in the past, which can be considered as the number of restarts. The more times the device node has been attempted, the less uncertainty there is in determining its status, and the lower the exploration entropy value. Conversely, the fewer times the device node has been attempted, the higher the uncertainty in determining its status, and the higher the exploration entropy value.

[0092] In a specific implementation, the network defense system can determine the number of times the device node has previously deployed the defense strategy, then determine the number of times the defense strategy is attempted (i.e., restarted) each time, and determine the exploration entropy value of the device node based on the number of attempts.

[0093] Step S305: Based on the updated estimate, the exploration entropy value, and the attacker information, determine whether to redeploy the defense strategy.

[0094] In a practical implementation, the aforementioned network defense system can use reinforcement learning algorithms to determine the current estimate of the device node by taking the updated estimate, exploration entropy value, and attacker information as input, and then redeploy the defense strategy based on the current estimate.

[0095] It should be understood that the current estimate mentioned above can be a measure of the degree of danger when a device node is currently under attack, after combining the updated estimate, the exploration entropy value, and attacker information.

[0096] Further, in this embodiment, step S305 includes:

[0097] Step S3051: Determine the abnormal information metric corresponding to the device node based on the attacker information.

[0098] It should be noted that the above-mentioned abnormal information metric can represent the probability that a device node is attacked by an attacker. The larger the abnormal information metric, the greater the probability that the device node is attacked by an attacker, and the more necessary it is to restart to reduce the risk.

[0099] In its implementation, the aforementioned network defense system can parse the attacker information provided by the honeypot node, determine the attacker's attack behavior, attack methods, and attack techniques, and determine the probability that the device node is attacked by the attacker based on the parsed information, and use this probability as an anomaly information metric.

[0100] Step S3052: Determine the current estimate of the device node based on the updated estimate, the exploration entropy value, and the anomaly information metric using the confidence interval upper bound algorithm, and determine the estimates of other device nodes in the network environment.

[0101] In a specific implementation, the reinforcement learning algorithm used by the aforementioned network defense system can be a confidence interval upper bound algorithm, which can add the updated estimate, exploration entropy value, and anomaly information metric as parameters to obtain the current estimate of the device node.

[0102] Step S3053: Compare the current estimate with the estimates of the other device nodes.

[0103] In a practical implementation, after determining the current estimate of the aforementioned device node, the network defense system can also calculate the current estimate of other device nodes based on the above method, and then compare the current estimate with the estimates of other device nodes.

[0104] Step S3054: When the current estimate is the maximum estimate, it is determined that there is no need to redeploy the defense strategy, which is a strategy to control the device node to perform defense actions.

[0105] In practical implementation, since the estimated value characterizes the degree of danger when a device node is attacked, the larger the estimated value, the more necessary it is to deploy a defense strategy and restart the device. This restart is the aforementioned defense action. Therefore, the network defense system can determine that the current estimated value is still at a relatively high level when compared with the estimated values ​​of other device nodes, i.e., the maximum estimated value. There is no need to redeploy the defense strategy; instead, the defense strategy is still deployed on the device node. Then, within a preset time step, the system controls the device node to perform a defense action based on the defense strategy, i.e., restart the device node, to defend against the attacker's attack and reduce security risks.

[0106] Furthermore, in this embodiment, after the step of determining that there is no need to redeploy the defense strategy when the current estimate is the maximum estimate, the method further includes:

[0107] Step S3055: Determine the number of currently available services in the network environment.

[0108] It should be noted that the number of available services in the current environment mentioned above refers to the number of available services in the network environment during this round of attack. This round of attack refers to the attack actions executed by the attacker at a preset time step.

[0109] In its implementation, after the aforementioned device nodes perform defensive actions, the network defense system can determine the number of available services in the current network environment.

[0110] Step S3056: Determine the network availability metric based on the number of available services in the current environment and the number of available services in the initial environment.

[0111] It should be noted that the number of services available in the initial environment mentioned above can be the number of services available in the initial environment provided during network environment initialization.

[0112] Understandably, if a network environment is attacked, the number of services initially available in that environment will decrease. The aforementioned network availability metric represents the ratio of currently usable services to the initial number of available services in the network environment.

[0113] In its implementation, the network defense system can obtain the network availability metric by dividing the number of available services in the current environment by the number of available services in the initial environment.

[0114] Step S3057: Determine whether the network availability metric has reached a preset metric threshold.

[0115] It should be noted that the above-mentioned preset measurement threshold can be used to determine whether the number of currently available network services can no longer support the normal operation of the system.

[0116] In its implementation, the aforementioned network defense system can determine in real time whether the network availability metric has reached the preset threshold within the current preset time step and between the next preset time step.

[0117] Step S3058: If yes, then return to the step of obtaining the reward information fed back by the network environment based on the defense strategy.

[0118] In its specific implementation, when the network availability metric is detected to be greater than the preset metric threshold, the network defense system can determine that the number of available network services can support the normal operation of the system, and thus determine that the defense strategy can defend against the attacker's attack after deployment. At this point, it can return to the operation of obtaining the reward information fed back by the network environment based on the defense strategy, repeat the above process, and continue the loop of the next time step to continuously adjust the defense strategy.

[0119] In addition, if the network availability metric is lower than the preset metric threshold, it will be determined whether the number of available network services can no longer support the normal operation of the system. The current round will end, the defender will be deemed to have failed, and the device information will be fed back to the user so that the user can investigate the anomaly and adjust the defense strategy.

[0120] Furthermore, in this embodiment, after step S3053, the method further includes:

[0121] Step S3054': When the current estimate is not the maximum estimate, select the target device node with the maximum estimate from the other device nodes.

[0122] In a specific implementation, when the network defense device detects that the current estimate of the device node is not the maximum estimate compared with the estimates of other device nodes, it determines that the device node has a lower degree of danger compared with other device nodes. At this time, the target device node with the maximum estimate, i.e. the highest degree of danger, can be selected from other device nodes.

[0123] Step S3055': Determine the attribute information of the target device node, and determine whether the target device node supports the deployment of the defense strategy based on the attribute information.

[0124] It should be noted that the above attribute information may be the device's configuration information, which includes information indicating whether the device node supports the deployment of defense strategies.

[0125] In practice, some device nodes may cause system interruption after restarting, making it impossible to continue providing services to users. Therefore, the network defense system can determine the attribute information of the target device node and then determine whether the target device node supports the deployment of defense policies based on the attribute information. In other words, it can determine whether the restart of the target device node will affect the operation of the system, so as to avoid the interruption of the system after the restart of device nodes that do not support the deployment of defense policies, and ensure that the system can provide services to users.

[0126] Step S3056': If supported, deploy the defense policy to the target device node so that the target device node performs the defense action based on the defense policy.

[0127] In its implementation, once the network defense system detects that the target device node supports the deployment of the aforementioned defense strategy, it can deploy the defense strategy to the target device node, enabling the target device node to perform defense actions based on the defense strategy. In other words, the restarted device node is switched to the target device node to resist the attacker's attack and reduce security risks.

[0128] It should be understood that if the target device nodes support the deployment of the above defense strategy, no defense action will be taken in this round.

[0129] For ease of understanding, please refer to Figure 5 This explanation does not limit the scope of this solution. Figure 5 This is a flowchart of the defense algorithm in the second embodiment of the network defense method of the present invention. Figure 5 In the process, the defender first performs initialization, initializing the parameters and data structures of the defender's agent, including setting the probability of each device node being selected to 1, setting the network availability metric to 1, and recording the number of times each device node is selected (the number of times the defense strategy is deployed). Then, it performs reward accumulation, accumulating the reward from the previous defense action's interaction with the environment over a period of time. Next, it performs scan frequency determination, checking if the defender has reached the preset time step for an action to be executed. Then, it performs an update estimate operation, updating the node's estimate based on the accumulated reward of the previous action. Finally, it performs action selection, using a reinforcement learning algorithm to select specific actions. The node with the highest estimated value is selected as the next action; then, the node redeployment algorithm is executed, which determines whether the selected device node algorithm can be redeployed based on the node's status and attributes; then, the action is executed. If the node is available for redeployment, the redeployment action (i.e., restart) is executed; if the node cannot be redeployed, the defending party does not perform any defense action; then, the actions and node termination are recorded, the executed actions are recorded and the node count is updated, and the network availability metric is queried; then, the environment interaction operation is performed. If the network availability metric is greater than or equal to the preset metric threshold, the loop continues to the next time step; otherwise, the current round ends, and the defense is deemed to have failed.

[0130] This embodiment determines the historical estimate of the device node, which is a measure of the degree of danger when the device node is attacked; it determines whether the current moment is at a preset time step that supports the redeployment of the defense strategy; if so, it updates the historical estimate according to the reward information to obtain the updated estimate; it determines the exploration entropy value of the device node based on the number of times the defense strategy has been deployed in the past; and it determines whether to redeploy the defense strategy based on the updated estimate, the exploration entropy value, and the attacker information. This embodiment uses a confidence interval upper bound algorithm, combined with the updated estimate, the exploration entropy value, and anomaly information measurement to determine the estimate of the device node, and redeploys the defense strategy based on this estimate, effectively improving the adjustment accuracy of the defense strategy.

[0131] refer to Figure 6 , Figure 6 This is a flowchart illustrating the third embodiment of the network defense method of the present invention.

[0132] Based on the above embodiments, in this embodiment, before step S20, the method further includes:

[0133] Step S21: Extract the environmental feature vector in the network environment.

[0134] It should be noted that the above environmental feature vectors can be vectors corresponding to features related to honeypot nodes, such as the services that honeypot nodes can provide, the ports that honeypot nodes have been active recently, the recent traffic of honeypot nodes, and the average network traffic of nodes around honeypot nodes.

[0135] In its implementation, the aforementioned network defense system can monitor network traffic and attacker behavior in the network environment in real time, and extract environmental feature vectors of the network environment based on the monitored network traffic and attack behavior.

[0136] Step S22: Calculate the expected reward value of different honeypot nodes based on the environmental feature vector using the linear confidence interval upper bound algorithm. The expected reward value represents the probability of each honeypot node being selected.

[0137] In a specific implementation, the network defense system described above can use the parameters in the environmental feature vector as input to the linear confidence interval upper bound algorithm. The weighted average of the parameters is then calculated using this algorithm as the context-aware reward expectation value of the honeypot node (referred to as reward expectation value for ease of understanding). The corresponding reward expectation value can be calculated for any honeypot node in this way.

[0138] It should be understood that the aforementioned expected reward value represents the probability of each honeypot node being selected. Since the network defense system does not know the attacker's target, it aims to find the honeypot node with the highest expected reward value for deployment through learning.

[0139] Step S23: Select the honeypot node to be deployed from the different honeypot nodes based on the expected reward value.

[0140] In its implementation, the aforementioned network defense system can select the honeypot node with the highest expected reward value from different honeypot nodes for deployment in the network environment. Furthermore, after the honeypot node with the highest expected reward value has been deployed for a period of time, other honeypot nodes that do not have the highest expected reward value can be randomly selected for deployment. The rewards reported by these honeypot nodes are then used to test the validity of the expected reward value calculation method.

[0141] Furthermore, in this embodiment, the method further includes:

[0142] Step S24: When the honeypot node needs to be switched, determine the attack damage information between the deployment time of the honeypot node and the current time.

[0143] In its specific implementation, when the network defense system detects that the current time is within the preset time step, it can determine which honeypot node can be switched, and the number of times it can determine the attack damage information fed back by the network environment between the time when the honeypot node started deployment and the current time.

[0144] Step S25: Update the environmental feature vector according to the attack damage information, and return to the step of calculating the expected reward value of different honeypot nodes based on the environmental feature vector using the linear confidence interval upper bound algorithm.

[0145] In a specific implementation, the network defense system can update the environmental feature vector based on the attack damage information, and then return to the operation of calculating the expected reward value of different honeypot nodes based on the environmental feature vector using the linear confidence interval upper bound algorithm. The above process is repeated to calculate the expected reward value of different honeypot nodes, so as to redeploy honeypot nodes based on environmental feedback, thereby more accurately estimating the defense effect of each honeypot node.

[0146] For ease of understanding, please refer to Figure 7 This explanation does not limit the scope of this solution. Figure 7 This is a flowchart of the honeypot proactive defense algorithm in the third embodiment of the network defense method of the present invention. Figure 7 In this process, the honeypot proactive defense first initializes parameters, setting initial expected reward values ​​and environmental feature vectors before starting to calculate the subsequent expected reward values ​​for each honeypot node. Then, it monitors network traffic and attack behavior, collecting information about acquisitions and attacks within the network. Next, it extracts feature vectors, extracting environmental feature vectors based on the monitored network traffic and attack behavior. Then, it calculates the context-aware expected reward value for each honeypot node, using a linear confidence interval upper bound algorithm to calculate the expected reward value for each honeypot node based on the environmental feature vectors. Next, it selects and deploys honeypot nodes, choosing the desired deployment based on the calculated expected reward values. Then, it monitors honeypot activity and attack behavior, collecting information about attacker reactions and attack methods after honeypot node deployment, feeding this information back to the aforementioned defense module. Finally, it updates the expected reward values ​​and feature vectors, updating them based on the monitored activity and attack behavior to more accurately estimate the effectiveness of each honeypot node. Finally, it returns to monitoring network traffic and attack behavior, repeating the above process to dynamically adjust the effectiveness of the honeypot nodes.

[0147] This embodiment extracts environmental feature vectors from the network environment; calculates the expected reward value for different honeypot nodes based on these feature vectors using a linear confidence interval upper bound algorithm, where the expected reward value represents the probability of each honeypot node being selected; and selects the honeypot nodes to be deployed from these different honeypot nodes based on the expected reward value. This embodiment selects honeypot nodes for deployment based on their expected reward values, which improves the deployment accuracy of honeypot nodes and allows them to provide more accurate attacker information, effectively enhancing the defense effectiveness of the defense strategy.

[0148] Furthermore, embodiments of the present invention also propose a storage medium storing a network defense program, which, when executed by a processor, implements the steps of the network defense method described above.

[0149] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the network defense device of the present invention.

[0150] like Figure 8 As shown, the network defense device proposed in this embodiment of the invention includes:

[0151] The reward acquisition module 501 is used to acquire reward information fed back by the network environment based on the defense strategy. The reward information is the attack loss information currently suffered after the deployment of the defense strategy.

[0152] The honeypot feedback module 502 is used to receive attacker information fed back by each honeypot node after it has induced an attacker to launch an attack.

[0153] The strategy deployment module 503 is used to determine whether to redeploy the defense strategy based on the reward information and the attacker information.

[0154] This embodiment obtains reward information from the network environment based on the defense strategy, where the reward information represents the current attack losses after the defense strategy has been deployed. It also receives attacker information from each honeypot node after inducing an attacker to launch an attack. Based on the reward information and the attacker information, it determines whether to redeploy the defense strategy. This embodiment comprehensively determines whether to redeploy the defense strategy by combining the reward information from the network environment and the attacker information from the honeypot nodes. Compared to the static defense strategies of existing technologies, this embodiment allows for adjustments based on both reward and attacker information after deployment, enabling flexible responses to constantly changing attack methods and effectively improving security defense efficiency.

[0155] Based on the first embodiment of the network defense device of the present invention described above, a second embodiment of the network defense device of the present invention is proposed.

[0156] In this embodiment, the strategy deployment module 503 is further configured to determine the historical estimate of the device node, wherein the historical estimate is a measure of the degree of danger when the device node is attacked; determine whether the current moment is at a preset time step that supports the redeployment of the defense strategy; if so, update the historical estimate according to the reward information to obtain the updated estimate; determine the exploration entropy value of the device node by the number of times the defense strategy has been deployed in the past; and determine whether to redeploy the defense strategy based on the updated estimate, the exploration entropy value, and the attacker information.

[0157] In one implementation, the policy deployment module 503 is further configured to: determine the anomaly information metric corresponding to the device node based on the attacker information; determine the current estimate of the device node based on the updated estimate, the exploration entropy value, and the anomaly information metric using a confidence interval upper bound algorithm, and determine the estimates of other device nodes in the network environment; compare the current estimate with the estimates of the other device nodes; and when the current estimate is the maximum estimate, determine that there is no need to redeploy the defense policy, wherein the defense policy is a strategy for controlling the device node to perform defense actions.

[0158] In one implementation, the policy deployment module 503 is further configured to determine the number of currently available services in the network environment; determine a network availability metric based on the current number of available services and the initial number of available services; determine whether the network availability metric has reached a preset metric threshold; and if so, perform the operation of obtaining reward information from the network environment based on the defense policy feedback.

[0159] In one implementation, the policy deployment module 503 is further configured to: select the target device node with the maximum estimate from the other device nodes when the current estimate is not the maximum estimate; determine the attribute information of the target device node; and determine whether the target device node supports the deployment of the defense policy based on the attribute information; if it supports the deployment, deploy the defense policy to the target device node so that the target device node performs the defense action based on the defense policy.

[0160] Based on the above embodiments of the network defense device of the present invention, a third embodiment of the network defense device of the present invention is proposed.

[0161] In this embodiment, the honeypot feedback module 502 is further configured to extract environmental feature vectors in the network environment; calculate the expected reward value of different honeypot nodes based on the environmental feature vectors using a linear confidence interval upper bound algorithm, wherein the expected reward value represents the probability of each honeypot node being selected; and select the honeypot node to be deployed from the different honeypot nodes based on the expected reward value.

[0162] In one implementation, the honeypot feedback module 502 is further configured to determine the attack damage information between the deployment time of the honeypot node and the current time when the honeypot node needs to be switched; update the environmental feature vector according to the attack damage information; and return the step of calculating the expected reward value of different honeypot nodes according to the environmental feature vector using the linear confidence interval upper bound algorithm.

[0163] The specific implementation of the network defense device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0164] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0165] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0167] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A network defense method, characterized in that, The method is applied to a security defense system deployed in a network environment, where device nodes and honeypot nodes are deployed. Both the device nodes and the honeypot nodes are targets of attackers. The honeypot nodes are nodes that lure attackers to launch attacks. The device nodes are equipped with defense strategies. The network defense methods include: Obtain reward information from the network environment based on the defense strategy, wherein the reward information is the attack loss information currently suffered after the deployment of the defense strategy; Receive attacker information fed back by each honeypot node after it has induced the attacker to attack; Determine whether to redeploy the defense strategy based on the reward information and the attacker information; The step of determining whether to redeploy the defense strategy based on the reward information and the attacker information includes: Determine the historical estimate of the device node, which is a measure of the effectiveness of the device node's defense against attacks under the defense strategy; Determine whether the current moment is within a preset time step that supports the redeployment of the defense strategy; If so, the historical estimate is updated based on the reward information to obtain the updated estimate; The exploration entropy value of the device node is determined by the number of times the defense strategy has been deployed in the past. Based on the updated estimate, the exploration entropy value, and the attacker information, a decision is made as to whether to redeploy the defense strategy.

2. The network defense method as described in claim 1, characterized in that, The step of determining whether to redeploy the defense strategy based on the updated estimate, the exploration entropy value, and the attacker information includes: Based on the attacker information, determine the abnormal information metric corresponding to the device node; The current estimate of the device node is determined by the confidence interval upper bound algorithm based on the updated estimate, the exploration entropy value, and the anomaly information metric, and the estimates of other device nodes in the network environment are also determined. Compare the current estimate with the estimates of the other device nodes; When the current estimate is the maximum estimate, it is determined that there is no need to redeploy the defense strategy, which is a strategy for controlling the device node to perform defense actions.

3. The network defense method as described in claim 2, characterized in that, After the step of determining that no redeployment of the defense strategy is needed when the current estimate is the maximum estimate, the method further includes: Determine the number of currently available services in the network environment; The network availability metric is determined based on the number of available services in the current environment and the number of available services in the initial environment. Determine whether the network availability metric has reached a preset metric threshold; If so, return to the step of obtaining the reward information fed back by the network environment based on the defense strategy.

4. The network defense method as described in claim 3, characterized in that, After the step of comparing the current estimate with the estimates of the other device nodes, the method further includes: When the current estimate is not the maximum estimate, the target device node with the maximum estimate is selected from the other device nodes; Determine the attribute information of the target device node, and determine whether the target device node supports the deployment of the defense strategy based on the attribute information; If supported, the defense strategy is deployed to the target device node so that the target device node performs the defense action based on the defense strategy.

5. The network defense method according to any one of claims 1 to 4, characterized in that, Before the step of receiving attacker information returned by each honeypot node after inducing the attacker to attack, the method further includes: Extract the environmental feature vector from the network environment; The expected reward value of different honeypot nodes is calculated based on the environmental feature vector using the linear confidence interval upper bound algorithm. The expected reward value represents the probability of each honeypot node being selected. Based on the expected reward value, the desired honeypot node is selected from the different honeypot nodes.

6. The network defense method as described in claim 5, characterized in that, The method further includes: When the honeypot node needs to be switched, determine the attack damage information between the deployment time of the honeypot node and the current time; Update the environmental feature vector based on the attack damage information, and return to the step of calculating the expected reward value of different honeypot nodes based on the environmental feature vector using the linear confidence interval upper bound algorithm.

7. A network defense device, characterized in that, The apparatus is used to implement the network defense method according to any one of claims 1 to 6, the apparatus comprising: The reward acquisition module is used to acquire reward information fed back by the network environment based on the defense strategy. The reward information is the attack loss information currently suffered after the deployment of the defense strategy. The honeypot feedback module is used to receive attacker information returned by each honeypot node after it has lured the attacker to attack. The strategy deployment module is used to determine whether to redeploy the defense strategy based on the reward information and the attacker information.

8. A network defense device, characterized in that, The device includes: a memory, a processor, and a network defense program stored in the memory and executable on the processor, the network defense program being configured to implement the steps of the network defense method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a network defense program, which, when executed by a processor, implements the steps of the network defense method as described in any one of claims 1 to 6.

Images