Attack surface evolution adaptive continuous governance system and method

Abstract

The embodiment of the disclosure provides an attack surface evolution adaptive and continuous management and control system and method, which are applied to the technical field of network security. The system comprises a monitoring agent, a decision-making agent, a convergence agent and a multi-agent collaborative strategy generation and optimization module. Based on the system, a multi-agent collaborative management and control mechanism can be constructed, a collaborative strategy among the monitoring agent, the decision-making agent and the convergence agent can be automatically generated, a complete management and control closed loop of "perception-decision-disposal-feedback" is formed, adaptive and continuous management and control of attack surface evolution is realized, the protection capability of the network system in response to various complex attacks is improved, and the safe and stable operation of the network system is ensured.

Description

Technical Field

[0001] This disclosure relates to the field of cybersecurity technology, and in particular to an attack surface evolution adaptive continuous management system and method. Background Technology

[0002] Currently, there are two main types of solutions for defending against cybersecurity attacks:

[0003] (1) A defense system based on fixed rules: It relies on a security rule base that is pre-configured manually, covering network access control rules, vulnerability signature databases, etc. When a network behavior or system status is detected to match the rule base, the corresponding defense action is triggered.

[0004] (2) Single agent defense mode: A single agent is used to collect limited-dimensional data from the system and use simple machine learning algorithms to directly generate defense decisions based on the monitoring results.

[0005] However, it's important to understand that in complex and dynamic environments, the attack surface evolution exhibits dynamism and uncertainty. The two approaches mentioned above struggle to adaptively adjust defense strategies for continuous dynamic control, and are unable to effectively address sudden network attacks and advanced persistent and covert attacks. Specifically, these issues manifest as follows:

[0006] (1) Weak ability to cope with dynamic attack surfaces: The fixed rule base relies on manual maintenance. When faced with the rapid evolution of the attack surface, the rule updates are lagging behind and cannot be adaptively adjusted; the data collection dimensions of a single intelligent agent are limited and the decision-making perspective is singular, making it difficult to fully perceive the complex attack surface state, resulting in a mismatch between the defense strategy and the actual attack surface evolution.

[0007] (2) Inadequate response to sudden attacks: Fixed rules cannot be identified due to the lack of predefined corresponding features. Due to the delay in data collection and decision-making and the simplicity of the strategy, a single intelligent agent cannot respond quickly and formulate an effective defense strategy, which makes the system vulnerable to being breached instantly.

[0008] (3) Inadequate defense against advanced persistent and covert attacks: Advanced persistent and covert attacks are characterized by long-term infiltration, multi-stage penetration, and strong behavioral camouflage. Fixed rules are insufficient to identify disguised attack behaviors; a single intelligent agent, lacking multi-stage coordination, cannot continuously track the evolution of the attack chain and is unable to effectively block it at each stage of the attack, resulting in the long-term lurking of the attack and harm to the system.

[0009] (4) Lack of multi-stage collaboration: Existing technologies have not established a deep collaboration mechanism among monitoring, decision-making, and convergence agents. Each stage is relatively independent, and the connection between information transmission and strategy execution is not smooth. Faced with complex attack surfaces, it is impossible to form a complete closed loop of "perception-decision-response", resulting in poor control efficiency and effectiveness, and the inability to achieve continuous dynamic control of the attack surface. Summary of the Invention

[0010] In a first aspect, embodiments of this disclosure provide an attack surface evolution adaptive continuous management and control system, the system comprising:

[0011] The module for generating and optimizing collaborative strategies for monitoring agents, decision-making agents, convergence agents, and multi-agent cooperative agents;

[0012] The monitoring agent is used to capture the attack surface state information of the network system in real time by using dynamic perception technology, combined with multi-dimensional data acquisition methods, and based on the state perception algorithm in a partially observable environment. It preprocesses and extracts features from the attack surface state information, obtains the attack surface state features, and transmits them to the decision agent.

[0013] The decision agent is used to perform strategy deduction in a complex dynamic environment model based on the attack surface state characteristics transmitted by the monitoring agent and an improved partially observable Markov decision algorithm. Based on the multi-agent cooperative reward mechanism, it generates decision instructions for the current attack surface state, determines the direction of defense and control, and feeds back the decision instructions to the monitoring agent to assist its subsequent state perception optimization. In addition, it also feeds back the decision instructions to the convergence agent.

[0014] The convergence agent is used to execute attack surface convergence operations based on the decision instructions fed back by the decision agent, and to feed back the execution results and the new network system status to the monitoring agent and the decision agent in real time, forming a closed-loop control process of "perception-decision-handling-feedback".

[0015] The multi-agent collaborative strategy generation and optimization module is used to construct an interactive communication mechanism and strategy collaborative optimization algorithm among the multi-agent agents by adopting an attack surface evolution adaptive continuous management and control model based on a multi-agent reinforcement learning framework. Through continuous interactive learning among the multi-agent agents, it automatically generates collaborative strategies among the monitoring agent, decision agent, and convergence agent, and dynamically adjusts the collaborative strategy parameters according to environmental feedback and management and control effects.

[0016] In some feasible examples of the first aspect, the model objectives and model architecture of the attack surface evolution adaptive continuous management model are as follows:

[0017] Model objective: Under partially observable conditions, through multi-agent collaboration, to track the dynamic evolution of the attack surface in real time, dynamically adjust defense strategies, and minimize the risk of attack surface exposure. This can be represented as: ,in This represents the attack surface state at time t. This represents the expected safety state at time t, where T represents the control period;

[0018] Model architecture: It adopts a hierarchical reinforcement learning framework, which is divided into an agent interaction layer, a policy learning layer, and an environment perception layer;

[0019] The algorithm design of the attack surface evolution adaptive continuous control model is shown below:

[0020] Let the attack surface state space be... , which is represented as ,in, , , The observation functions of the monitoring agent represent network connection status, vulnerability exposure status, and permission status, respectively. By analyzing the attack surface state space The results are obtained through multi-dimensional sampling and feature extraction, satisfying... , which is represented as ,in This represents the parameters of the observation function.

[0021] In some implementable examples of the first aspect, the observation strategy of the monitoring agent is: observation update based on Bayesian filtering, and observation probability. ,in Indicates a normal distribution. Represents the variance of observation noise. This indicates the attack surface observation results. To represent the true state of the attack surface, by maximizing the observation likelihood. Optimize observation function parameters .

[0022] In some implementable examples of the first aspect, the decision agent employs the Deep Deterministic Policy Gradient (DDPG) algorithm, whose policy network is represented as follows: Based on the state transmitted by the monitoring agent Output decision action , Let the set of network parameters of the decision-making agent be denoted as and the value function be denoted as . It is used to evaluate the value of a decision action in the current state and to satisfy... ,in Represents the mathematical expectation. Indicates a reward. Indicates the discount factor. This represents the objective value function of the decision-making agent. This represents the output of the decision-making agent's goal-policy network. This represents the set of network parameters for the decision-making agent's objective policy. Indicates the state at the next moment. This represents the set of parameters for the value function of the decision-making agent.

[0023] In some implementable examples of the first aspect, the policy network of the convergent agent is represented as: Receive action instructions and status from the decision-making agent Output convergence action , Let represent the set of parameters of the convergent agent policy network, and let the value function be expressed as . Used to evaluate the value of convergence actions. This represents the set of parameters for the value function of a convergent agent.

[0024] In some implementable examples of the first aspect, the multi-agent cooperative reward mechanism is represented as follows:

[0025] ,in Indicates the total reward. This indicates the attack surface convergence reward. This indicates a reward for responding to a sudden attack. This indicates a reward for covert attack tracking. This indicates a penalty for resource consumption. This represents the reward weighting coefficient.

[0026] In some implementable examples of the first aspect, the multi-agent policy cooperative optimization algorithm is expressed as:

[0027] Based on a centralized training-distributed execution framework, the parameters of the policy network and value function are updated and optimized using empirical data from all agents; the optimization objective is to minimize the mean squared error between the value function and the target value function. , represented as Where E represents the expected value, This represents the current value function of the agent. Indicates an immediate reward. Indicates the discount factor. This represents the output of the objective value function. This represents the set of parameters for the objective value function; and is expressed through the policy gradient. Update the policy network parameters, where E represents the mathematical expectation. Represents the partial derivative of the action. This represents the current value function of the agent. This indicates the action output by the policy network. The partial derivatives of the policy network parameters are represented. This represents the current agent policy network.

[0028] Secondly, embodiments of this disclosure provide an attack surface evolution adaptive continuous management method, which is applied to the system described above, including:

[0029] The monitoring agent uses dynamic perception technology, combined with multi-dimensional data acquisition methods, and based on state perception algorithms in partially observable environments, to capture the attack surface state information of the network system in real time. It preprocesses and extracts features from the attack surface state information, obtains the attack surface state features, and transmits them to the decision agent.

[0030] Based on the attack surface state characteristics transmitted by the monitoring agent, the decision agent uses an improved partially observable Markov decision algorithm to perform strategy deduction in a complex dynamic environment model. According to the multi-agent cooperative reward mechanism, it generates decision instructions for the current attack surface state, determines the direction of defense and control, and feeds back the decision instructions to the monitoring agent to assist its subsequent state perception optimization. In addition, it also feeds back the decision instructions to the convergence agent.

[0031] The convergence agent executes attack surface convergence operations based on the decision instructions fed back by the decision agent, and feeds back the execution results and the new network system status to the monitoring agent and the decision agent in real time, forming a closed-loop control process of "perception-decision-handling-feedback".

[0032] The multi-agent collaborative strategy generation and optimization module adopts an attack surface evolution adaptive continuous management and control model based on a multi-agent reinforcement learning framework. It constructs an interactive communication mechanism and a strategy collaborative optimization algorithm among the multi-agents. Through continuous interactive learning among the multi-agents, it automatically generates collaborative strategies among the monitoring agent, decision agent, and convergence agent. Based on environmental feedback and management and control effects, it dynamically adjusts the collaborative strategy parameters.

[0033] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0034] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0035] In this embodiment, the attack surface evolution adaptive continuous management and control system includes: a monitoring agent, a decision agent, a convergence agent, and a multi-agent collaborative strategy generation and optimization module. Based on this system, a multi-agent collaborative management and control mechanism can be constructed, and collaborative strategies among the monitoring agent, decision agent, and convergence agent can be automatically generated to form a complete management and control closed loop of "perception-decision-handling-feedback". This enables adaptive continuous management and control of attack surface evolution, improves the network system's ability to defend against various complex attacks, and ensures the safe and stable operation of the network system.

[0036] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0037] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0038] Figure 1 This diagram illustrates an architecture of an attack surface evolution adaptive continuous management system provided in an embodiment of this disclosure.

[0039] Figure 2 A flowchart of an attack surface evolution adaptive continuous management method provided in an embodiment of this disclosure is shown;

[0040] Figure 3 A structural diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0042] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0043] To address the problems identified in the background technology, this disclosure provides an adaptive and continuous management system, method, device, and storage medium for attack surface evolution. Specifically, it can construct a multi-agent collaborative management mechanism, automatically generate collaborative strategies among monitoring agents, decision-making agents, and convergence agents, forming a complete management closed loop of "perception-decision-handling-feedback," achieving adaptive and continuous management of attack surface evolution, improving the network system's ability to defend against various complex attacks, and ensuring the safe and stable operation of the network system.

[0044] The following detailed description, with reference to the accompanying drawings, illustrates a system, method, device, and storage medium for attack surface evolution adaptive continuous management provided in this disclosure through specific embodiments.

[0045] Figure 1 This illustration shows an architecture diagram of an attack surface evolution adaptive continuous management system provided in an embodiment of this disclosure, such as... Figure 1 As shown, system 100 may include: a monitoring agent, a decision agent, a convergence agent, and a multi-agent collaborative strategy generation and optimization module.

[0046] The monitoring agent employs dynamic sensing technology, combined with multi-dimensional data collection methods such as full-dimensional network traffic collection, in-depth analysis of system process behavior, and user operation auditing. Based on state perception algorithms under partially observable environments, it captures attack surface state information of the network system in real time, preprocesses and extracts features from the attack surface state information, obtains attack surface state features, and transmits them to the decision agent. This provides the decision agent with comprehensive and accurate attack surface state feature input, especially enhancing the early state capture of sudden and covert attacks.

[0047] The decision agent is used to perform strategy deduction in a complex dynamic environment model based on the attack surface state characteristics transmitted by the monitoring agent and using an improved partially observable Markov decision algorithm. Based on the multi-agent cooperative reward mechanism, it generates decision instructions for the current attack surface state, determines the direction of defense and control, and feeds back the decision instructions to the monitoring agent to assist its subsequent state perception optimization. In addition, the decision instructions are also fed back to the convergence agent.

[0048] The convergence agent is used to execute attack surface convergence operations based on the decision instructions fed back by the decision agent. For example, in the case of sudden attacks, it can quickly perform traffic scrubbing and attack source blocking; in the case of advanced persistent and covert attacks, it can implement multi-stage responses and feed back the execution results and the new network system status to the monitoring agent and the decision agent in real time, forming a closed-loop control process of "perception-decision-handling-feedback" to ensure continuous tracking and dynamic control of the attack surface evolution.

[0049] The multi-agent collaborative strategy generation and optimization module is used to construct an interactive communication mechanism and strategy collaborative optimization algorithm among the multi-agent agents. Through continuous interactive learning among the multi-agent agents, it automatically generates collaborative strategies among the monitoring agent, decision-making agent, and convergence agent. Based on environmental feedback and control effects, it dynamically adjusts the collaborative strategy parameters to ensure efficient collaboration among agents in complex scenarios such as sudden attacks and advanced persistent covert attacks, thereby achieving adaptive and continuous control over the attack surface evolution.

[0050] In some embodiments, the model objective and model architecture of the attack surface evolution adaptive continuous management model are as follows:

[0051] Model objective: Under partially observable conditions, through multi-agent collaboration, to track the dynamic evolution of the attack surface in real time, dynamically adjust defense strategies, and minimize the risk of attack surface exposure. This can be represented as: ,in This represents the attack surface state at time t. This represents the desired security state at time t, where T represents the control period, enabling rapid response to sudden attacks and continuous tracking and control of covert attacks.

[0052] Model architecture: It adopts a hierarchical reinforcement learning framework, which is divided into an agent interaction layer, a policy learning layer, and an environment perception layer.

[0053] In some embodiments, the algorithm design of the attack surface evolution adaptive continuous management model is as follows:

[0054] Let the attack surface state space be... It encompasses subspaces such as network connection status, vulnerability exposure status, and permission status, and is represented as follows: ,in, , , The observation functions of the monitoring agent represent network connection status, vulnerability exposure status, and permission status, respectively. By analyzing the attack surface state space The results are obtained through multi-dimensional sampling and feature extraction, satisfying... , which is represented as ,in The parameters of the observation function are optimized through pre-training or online learning to improve the ability to detect sudden and covert attacks.

[0055] In some embodiments, the observation strategy of the monitoring agent is: observation update based on Bayesian filtering, and observation probability. ,in Indicates a normal distribution. Represents the variance of observation noise. This indicates the attack surface observation results. To represent the true state of the attack surface, by maximizing the observation likelihood. Optimize observation function parameters This enhances the ability to identify covert attacks.

[0056] In some embodiments, the decision agent employs the Deep Deterministic Policy Gradient (DDPG) algorithm, whose policy network is represented as follows: Based on the state transmitted by the monitoring agent Output decision action , Let the set of network parameters of the decision-making agent be denoted as and the value function be denoted as . It is used to evaluate the value of a decision action in the current state and to satisfy... ,in Represents the mathematical expectation. Indicates a reward. Indicates the discount factor. This represents the objective value function of the decision-making agent. This represents the output of the decision-making agent's goal-policy network. This represents the set of network parameters for the decision-making agent's objective policy. Indicates the state at the next moment. This represents the set of parameters for the value function of a decision-making agent. The value function guides the decision-making agent to optimize its strategy, thereby improving the quality of decision-making in response to sudden and covert attacks.

[0057] In some embodiments, the policy network of the converging agent is represented as: Receive action instructions and status from the decision-making agent Output convergence action , Let represent the set of parameters of the convergent agent policy network, and let the value function be expressed as . Used to evaluate the value of convergence actions. This represents the set of parameters for the value function of the convergent agent. The update method is similar to that of the decision agent. Through collaborative optimization, it ensures rapid response to sudden attacks and continuous convergence against covert attacks.

[0058] In some embodiments, the multi-agent cooperative reward mechanism is represented as follows:

[0059] ,in Indicates the total reward. This indicates the attack surface convergence reward. This indicates a reward for responding to a sudden attack. This indicates a reward for covert attack tracking. This indicates a penalty for resource consumption. This represents the reward weighting coefficient, which is dynamically adjusted based on system security requirements and resource constraints. Specifically:

[0060] This is used to measure the overall reduction in the attack surface, and is effective for convergence at all stages of sudden and covert attacks. for Attack surface status at all times. for Attack surface status at all times.

[0061] It is used to incentivize decision-making agents to make rapid decisions, converge agents to execute efficiently, and respond to sudden attacks. Rewards are given for successfully responding to sudden attacks.

[0062] This is used to encourage agents to continuously track covert attack chains, where In order to conduct covert attacks Rewards for successfully identifying and blocking a stage. Indicates the number of stages.

[0063] The computational and time resource consumption of the convergence action is positively correlated, thus avoiding excessive consumption of system resources.

[0064] In some embodiments, the multi-agent policy cooperative optimization algorithm is expressed as:

[0065] Based on a centralized training-distributed execution framework, the parameters of the policy network and value function are updated and optimized using empirical data from all agents; the optimization objective is to minimize the mean squared error between the value function and the target value function. , represented as Where E represents the expected value, This represents the current value function of the agent. Indicates an immediate reward. Indicates the discount factor. This represents the output of the objective value function. This represents the set of parameters for the objective value function; and is expressed through the policy gradient. Update the policy network parameters to achieve multi-agent collaborative policy optimization and adapt to dynamic attack surface evolution, where E represents the mathematical expectation. Represents the partial derivative of the action. This represents the current value function of the agent. This indicates the action output by the policy network. The partial derivatives of the policy network parameters are represented. This represents the current agent policy network.

[0066] In summary, this disclosure achieves at least the following technical effects:

[0067] The attack surface evolution-based adaptive continuous management and control system can build a multi-agent collaborative management and control mechanism, automatically generate collaborative strategies among monitoring agents, decision-making agents, and convergence agents, and form a complete management and control closed loop of "perception-decision-response-feedback". This enables adaptive continuous management and control of attack surface evolution, improves the network system's ability to cope with various complex attacks, and ensures the safe and stable operation of the network system.

[0068] The above is an introduction to the system embodiments. The following method embodiments will further illustrate the solution described in this disclosure.

[0069] Figure 2 A flowchart of an attack surface evolution adaptive persistent management method provided in an embodiment of this disclosure is shown, such as... Figure 2 As shown, method 200 is applied to system 100 as described above, including:

[0070] S210, the monitoring agent adopts dynamic perception technology, combined with multi-dimensional data acquisition methods, and based on the state perception algorithm in a partially observable environment, captures the attack surface state information of the network system in real time, preprocesses and extracts features from the attack surface state information, obtains the attack surface state features, and transmits them to the decision agent.

[0071] S220: Based on the attack surface state characteristics transmitted by the monitoring agent, the decision agent uses an improved partially observable Markov decision algorithm to perform strategy deduction in a complex dynamic environment model. According to the multi-agent cooperative reward mechanism, it generates decision instructions for the current attack surface state, determines the direction of defense and control, and feeds back the decision instructions to the monitoring agent to assist its subsequent state perception optimization. In addition, it also feeds back the decision instructions to the convergence agent.

[0072] S230: The convergence agent executes attack surface convergence operations based on the decision instructions fed back by the decision agent, and feeds back the execution results and the new network system status to the monitoring agent and the decision agent in real time, forming a closed-loop control process of "perception-decision-handling-feedback".

[0073] S240, the multi-agent cooperative strategy generation and optimization module adopts an attack surface evolution adaptive continuous management and control model based on a multi-agent reinforcement learning framework. It constructs an interactive communication mechanism and a strategy cooperative optimization algorithm among the multi-agents. Through continuous interactive learning among the multi-agents, it automatically generates cooperative strategies among the monitoring agent, the decision agent, and the convergence agent. Based on environmental feedback and management and control effects, it dynamically adjusts the cooperative strategy parameters.

[0074] It is understandable that each step in the above method corresponds to the function of each part of the above system and can achieve its corresponding technical effect. For the sake of brevity, it will not be elaborated here.

[0075] Figure 3A structural diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Electronic device 300 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 300 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0076] like Figure 3 As shown, the electronic device 300 may include a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0077] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0078] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer program product, including a computer program tangibly contained in a computer-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform method 200 by any other suitable means (e.g., by means of firmware).

[0079] The various embodiments described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0080] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0081] In the context of this disclosure, a computer-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0082] It should be noted that this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute method 200 and achieve the corresponding technical effects achieved by executing the method in the embodiments of this disclosure. For the sake of brevity, they will not be described in detail here.

[0083] In addition, this disclosure also provides a computer program product including a computer program that implements method 200 when executed by a processor.

[0084] To provide interaction with a user, the embodiments described above can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0085] The embodiments described above can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with the implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0086] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0087] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0088] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An attack surface evolution adaptive continuous control system, characterized in that, The system includes: The module for generating and optimizing collaborative strategies for monitoring agents, decision-making agents, convergence agents, and multi-agent cooperative agents; The monitoring agent is used to capture the attack surface state information of the network system in real time by using dynamic perception technology, combined with multi-dimensional data acquisition methods, and based on the state perception algorithm in a partially observable environment. It preprocesses and extracts features from the attack surface state information, obtains the attack surface state features, and transmits them to the decision agent. The decision agent is used to perform strategy deduction in a complex dynamic environment model based on the attack surface state characteristics transmitted by the monitoring agent and an improved partially observable Markov decision algorithm. Based on the multi-agent cooperative reward mechanism, it generates decision instructions for the current attack surface state, determines the direction of defense and control, and feeds back the decision instructions to the monitoring agent to assist its subsequent state perception optimization. In addition, it also feeds back the decision instructions to the convergence agent. The convergence agent is used to execute attack surface convergence operations based on the decision instructions fed back by the decision agent, and to feed back the execution results and the new network system status to the monitoring agent and the decision agent in real time, forming a closed-loop control process of "perception-decision-handling-feedback". The multi-agent cooperative strategy generation and optimization module is used to construct an attack surface evolution adaptive continuous management and control model based on a multi-agent reinforcement learning framework. It constructs an interactive communication mechanism and a strategy cooperative optimization algorithm among the multi-agents. Through continuous interactive learning among the multi-agents, it automatically generates cooperative strategies among the monitoring agent, decision agent, and convergence agent. Based on environmental feedback and management and control effects, it dynamically adjusts the cooperative strategy parameters.

2. The system according to claim 1, characterized in that, The model objectives and architecture of the attack surface evolution adaptive continuous control model are shown below: Model objective: Under partially observable conditions, through multi-agent collaboration, to track the dynamic evolution of the attack surface in real time, dynamically adjust defense strategies, and minimize the risk of attack surface exposure. This can be represented as: ,in This represents the attack surface state at time t. This represents the expected safety state at time t, where T represents the control period; Model architecture: It adopts a hierarchical reinforcement learning framework, which is divided into an agent interaction layer, a policy learning layer, and an environment perception layer; The algorithm design of the attack surface evolution adaptive continuous control model is shown below: Let the attack surface state space be... , which is represented as ,in, , , The observation functions of the monitoring agent represent network connection status, vulnerability exposure status, and permission status, respectively. By analyzing the attack surface state space The results are obtained through multi-dimensional sampling and feature extraction, satisfying... , which is represented as ,in This represents the parameters of the observation function.

3. The system according to claim 2, characterized in that, The monitoring agent's observation strategy is: Bayesian filtering-based observation update, with observation probabilities... ,in Indicates a normal distribution. Represents the variance of observation noise. This indicates the attack surface observation results. To represent the true state of the attack surface, by maximizing the observation likelihood. Optimize observation function parameters .

4. The system according to claim 3, characterized in that, The decision agent employs the Deep Deterministic Policy Gradient (DDPG) algorithm, whose policy network is represented as follows: Based on the state transmitted by the monitoring agent Output decision action , Let the set of network parameters of the decision-making agent be denoted as and the value function be denoted as . It is used to evaluate the value of a decision action in the current state and to satisfy... ,in Represents the mathematical expectation. Indicates a reward. Indicates the discount factor. This represents the objective value function of the decision-making agent. This represents the output of the decision-making agent's goal-policy network. This represents the set of network parameters for the decision-making agent's objective policy. Indicates the state at the next moment. This represents the set of parameters for the value function of the decision-making agent.

5. The system according to claim 4, characterized in that, The policy network of the convergent agent is represented as Receive action instructions and status from the decision-making agent Output convergence action , Let represent the set of parameters of the convergent agent policy network, and let the value function be expressed as . Used to evaluate the value of convergence actions. This represents the set of parameters for the value function of a convergent agent.

6. The system according to claim 5, characterized in that, The multi-agent cooperative reward mechanism is represented as follows: ,in Indicates the total reward. This indicates the attack surface convergence reward. This indicates a reward for responding to a sudden attack. This indicates a reward for covert attack tracking. This indicates a penalty for resource consumption. This represents the reward weighting coefficient.

7. The system according to claim 6, characterized in that, The multi-agent policy cooperative optimization algorithm is expressed as: Based on a centralized training-distributed execution framework, the parameters of the policy network and value function are updated and optimized using empirical data from all agents; the optimization objective is to minimize the mean squared error between the value function and the target value function. , represented as Where E represents the expected value, This represents the current value function of the agent. Indicates a reward. Indicates the discount factor. This represents the output of the objective value function. This represents the set of parameters for the objective value function; and is expressed through the policy gradient. Update the policy network parameters, where E represents the mathematical expectation. Represents the partial derivative of the action. This represents the current value function of the agent. This indicates the action output by the policy network. The partial derivatives of the policy network parameters are represented. This represents the current agent policy network.

8. A method for adaptive and continuous control of attack surface evolution, characterized in that, The method is applied to the system according to any one of claims 1-7, comprising: The monitoring agent uses dynamic perception technology, combined with multi-dimensional data acquisition methods, and based on state perception algorithms in partially observable environments, to capture the attack surface state information of the network system in real time. It preprocesses and extracts features from the attack surface state information, obtains the attack surface state features, and transmits them to the decision agent. Based on the attack surface state characteristics transmitted by the monitoring agent, the decision agent uses an improved partially observable Markov decision algorithm to perform strategy deduction in a complex dynamic environment model. According to the multi-agent cooperative reward mechanism, it generates decision instructions for the current attack surface state, determines the direction of defense and control, and feeds back the decision instructions to the monitoring agent to assist its subsequent state perception optimization. In addition, it also feeds back the decision instructions to the convergence agent. The convergence agent executes attack surface convergence operations based on the decision instructions fed back by the decision agent, and feeds back the execution results and the new network system status to the monitoring agent and the decision agent in real time, forming a closed-loop control process of "perception-decision-handling-feedback". The multi-agent collaborative strategy generation and optimization module adopts an attack surface evolution adaptive continuous management and control model based on a multi-agent reinforcement learning framework. It constructs an inter-agent interactive communication mechanism and a strategy collaborative optimization algorithm. Through continuous interactive learning among multiple agents, it automatically generates collaborative strategies among monitoring agents, decision-making agents, and convergence agents. Based on environmental feedback and management effectiveness, it dynamically adjusts the collaborative strategy parameters.

9. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of claim 8.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method of claim 8.

Images