AI agent security threat handling-self-healing collaborative integration system fusing knowledge graph and reinforcement learning

Abstract

The present application relates to the field of artificial intelligence technology, specifically, to an AI agent security threat disposal-self-healing collaborative integration system fusing knowledge graph and reinforcement learning, comprising: a security threat knowledge graph construction unit; a security situation awareness and threat identification unit; a disposal-self-healing decision generation unit; a threat disposal and self-healing execution unit; a collaborative control and iterative optimization unit.The present application extends the application of the fusion of knowledge graph and reinforcement learning to the whole link of AI agent security threat disposal and self-healing in industrial systems, and constructs a coupled action space of disposal action and self-healing action, realizes the synchronous execution of threat disposal operation and AI agent hierarchical self-healing operation, simultaneously takes the security threat knowledge graph as the prior constraint of reinforcement learning, and can also reversely update the knowledge graph entity and the associated relationship through the execution feedback, forming a bidirectional fusion application system of knowledge graph and reinforcement learning.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an integrated system for handling and self-healing collaborative security threats to AI agents that integrates knowledge graphs and reinforcement learning. Background Technology

[0002] In the operation of various industrial systems, AI agents face complex and ever-changing security threats, which places stringent demands on the efficiency of rapid response to security threats and the autonomous self-healing capabilities of AI agents. The technical approach of integrating knowledge graphs and reinforcement learning has become a core development direction for improving the security protection level of AI agents.

[0003] In existing technologies, relevant patents have been researching the integration of knowledge graphs and reinforcement learning in the field of cybersecurity threat analysis. For example, invention patent CN202411539104.X discloses a method and system for discovering network attack paths based on knowledge graphs. It extracts entities, attributes, and relationships from network attack and defense data to construct a knowledge graph, obtains its vector representation through the TransE model, and then uses this representation as the interaction environment for a reinforcement learning agent. This transforms attack path prediction into a sequence decision problem, predicting attack paths through interactive reasoning between the agent and the environment, thus assisting in the formulation of defense measures. Another example is invention patent CN202510273574.4, which discloses a threat intelligence knowledge graph reasoning method based on improved GNN and reinforcement learning. It introduces a variational mechanism to construct an improved GNN autoencoder to extract threat intelligence knowledge graph features, builds a policy network with a soft reward module, and updates parameters using a policy gradient method. This completes the construction of a knowledge reasoning model integrating improved GNN and reinforcement learning and outputs the reasoning results, improving feature extraction efficiency and the accuracy and interpretability of reasoning.

[0004] Despite the design advantages of the aforementioned technical solutions, they also suffer from the following technical shortcomings: First, the integration of knowledge graphs and reinforcement learning is limited to the threat reasoning and prediction stage, without extending to the actual operation of security handling and AI agent self-healing, or the linkage between the two: Invention patent CN202411539104.X integrates the two technologies to only achieve network attack path reasoning and prediction, which can only assist in formulating defense measures, without actual security threat handling operations, let alone AI agent self-healing operations; the integration scheme of CN202510273574.4 also only focuses on knowledge reasoning of threat intelligence, without being implemented in the actual execution stages of security handling and AI agent self-healing. Secondly, the fusion solutions are only suitable for a single scenario of cybersecurity threat inference and do not meet the actual security protection needs of AI agents: the fusion solutions of invention patents CN202411539104.X and CN202510273574.4 are both designed around threat inference and prediction targets in the field of cybersecurity, without considering the actual characteristics of AI agent operation. Their solutions lack a full-process design for AI agent security protection and cannot be directly applied to actual security protection scenarios for AI agents. Therefore, we propose an integrated AI agent security threat handling and self-healing collaborative system that integrates knowledge graphs and reinforcement learning. Summary of the Invention

[0005] The purpose of this invention is to provide an integrated system for security threat handling and self-healing of AI agents that combines knowledge graphs and reinforcement learning. This system addresses the problem in the background art where the integration of knowledge graphs and reinforcement learning is limited to the threat reasoning and prediction stage, without extending to the actual operation of security handling and AI agent self-healing, as well as the linkage between the two. Furthermore, the design of the integration scheme is only suitable for the single scenario of network security threat reasoning and does not meet the security protection needs of the actual operation of AI agents.

[0006] To address the aforementioned technical problems, the present invention aims to provide an integrated system for handling and self-healing collaborative security threats against AI agents, combining knowledge graphs and reinforcement learning, comprising:

[0007] The security threat knowledge graph construction unit collects AI agent operation data, security alarm data, vulnerability detection data, and AI agent business capability baseline data. Through named entity recognition, entity relationship alignment, and ontology construction technology, it constructs a security threat knowledge graph containing core entities and related relationships, including agent nodes, business links, security threats, vulnerabilities, and business capability baselines. The security threat knowledge graph data is then output to the security situation awareness and threat identification unit.

[0008] The security situation awareness and threat identification unit receives security threat knowledge graph data output by the security threat knowledge graph construction unit, performs entity relationship reasoning based on entity relationship in the security threat knowledge graph data, completes real-time security situation awareness and threat identification of AI agent, generates and outputs standardized threat relationship data, AI agent business capability baseline data, business link and security status relationship data, and synchronously transmits them to the handling-self-healing decision generation unit.

[0009] The handling-self-healing decision generation unit receives standardized threat association data, AI agent business capability baseline data, and business link and security status association data output by the security situation awareness and threat identification unit. Using a security threat knowledge graph as a priori constraint for reinforcement learning, it constructs a reinforcement learning state space based on threat status, vulnerability status, and the degree of business capability impairment. Based on the business link and security status association rules, it constructs an action space coupling handling actions and self-healing actions, eliminating actions that would cause irreversible damage to the AI ​​agent's business capabilities. It constructs a multi-objective reward and punishment function with threat handling success rate, business capability recovery rate, system self-healing success rate, and business interruption duration as optimization objectives. Through reinforcement learning, it generates security handling strategies and hierarchical self-healing action sequences adapted to the current threat. Based on the business capability baseline, it verifies the impact of the action sequences on business capabilities; if a preset verification threshold is not reached, it iterative optimization is performed, and the verified security handling strategies and self-healing action sequences are output to the threat handling and self-healing execution unit.

[0010] The threat handling and self-healing execution unit receives the verified security handling strategy and self-healing action sequence output by the handling-self-healing decision generation unit, executes security threat handling operations such as attack blocking and malicious code removal, and simultaneously executes AI agent hierarchical self-healing operations such as configuration restoration and node repair. During the execution process, it verifies the business capability status against the AI ​​agent business capability baseline in real time, collects handling result data, self-healing result data, and business capability status data, forms full-process execution feedback data, and outputs it to the collaborative management and iterative optimization unit.

[0011] The collaborative management and iterative optimization unit receives full-process execution feedback data output by the threat handling and self-healing execution unit. Based on the security threat knowledge graph output by the security threat knowledge graph construction unit, it implements threat intelligence synchronization among multiple AI agents, coordinated action linkage, and unified scheduling of self-healing resources. According to the execution feedback data, it iteratively optimizes the decision generation process of the handling-self-healing decision generation unit based on reinforcement learning, updates the entity and relationship data of the security threat knowledge graph, and feeds back the updated security threat knowledge graph data to the security threat knowledge graph construction unit.

[0012] As a further improvement to this technical solution, the security threat knowledge graph construction unit includes a data acquisition module, a data association processing module, an ontology construction module, and a graph generation and output module, wherein:

[0013] The data acquisition module collects AI agent operation data, security alarm data, vulnerability detection data, and AI agent business capability baseline data.

[0014] The data association processing module performs named entity recognition and entity relationship alignment on the data collected by the data acquisition module.

[0015] The ontology construction module constructs core entities and their relationships, including intelligent agent nodes, business links, security threats, vulnerabilities, and business capability baselines, based on the processing results of the data association processing module.

[0016] The graph generation and output module generates a security threat knowledge graph based on the construction results of the ontology construction module, and outputs the security threat knowledge graph data to the security situation awareness and threat identification unit.

[0017] As a further improvement to this technical solution, the security situation awareness and threat identification unit includes a data receiving and parsing module, an entity association reasoning module, a situation awareness and threat identification module, and a data generation and output module, wherein:

[0018] The data receiving and parsing module receives the security threat knowledge graph data output by the security threat knowledge graph construction unit, and performs format parsing and validity verification on the security threat knowledge graph data.

[0019] The entity association reasoning module performs entity association reasoning based on the entity associations in the security threat knowledge graph data parsed by the data receiving and parsing module, and outputs the entity association reasoning results.

[0020] The situational awareness and threat identification module receives the entity association reasoning results output by the entity association reasoning module and completes real-time security situational awareness and threat identification for the AI ​​agent.

[0021] The data generation and output module generates and outputs standardized threat-related data, AI agent business capability baseline data, and business link and security status-related data based on the identification results of the situational awareness and threat identification module, and transmits them synchronously to the handling-self-healing decision generation unit.

[0022] As a further improvement to this technical solution, the disposal-self-healing decision generation unit includes a data receiving module, a state space construction module, an action space construction and filtering module, a reward and punishment function construction module, a decision generation and output module, and a verification and optimization module, wherein:

[0023] The data receiving module receives standardized threat correlation data, AI agent business capability baseline data, and business link and security status correlation data output by the security situation awareness and threat identification unit.

[0024] The state space construction module uses the security threat knowledge graph as a prior constraint for reinforcement learning to construct the reinforcement learning state space. ;

[0025] The action space construction and filtering module constructs and filters the action space coupled with handling actions and self-healing actions based on the association rules between business links and security status.

[0026] The reward and punishment function construction module constructs a multi-objective adaptive reward and punishment function based on a preset optimization objective. ;

[0027] The decision generation and output module generates security handling strategies and hierarchical self-healing action sequences adapted to the current threat through reinforcement learning, and transmits the security handling strategies and hierarchical self-healing action sequences to the verification and optimization module.

[0028] The verification and optimization module verifies and iteratively optimizes the effectiveness of the security handling strategy and the business security of the hierarchical self-healing action sequence based on the baseline of the AI ​​agent's business capabilities and the requirements for handling security threats. After the verification is passed, the decision generation and output module outputs the security handling strategy and the hierarchical self-healing action sequence to the threat handling and self-healing execution unit.

[0029] As a further improvement to this technical solution, the state space construction module uses the security threat knowledge graph as a prior constraint for reinforcement learning to construct the reinforcement learning state space. The process includes the following steps:

[0030] S32.1. Based on the entity relationships in the security threat knowledge graph, determine the three core state dimensions: threat, vulnerability, and business impairment.

[0031] S32.2 Constructing a reinforcement learning state space based on three core state dimensions. ;

[0032] S32.3, Strengthen the learning state space The output is sent to the decision generation and output module as the state input for reinforcement learning.

[0033] As a further improvement to this technical solution, the process of constructing and filtering the action space based on the association rules between business links and security status, and constructing and filtering the action space coupled with handling actions and self-healing actions, includes the following steps:

[0034] S33.1 Establish the linkage and coupling logic between treatment actions and self-healing actions, and clarify the timing and correlation of action combinations;

[0035] S33.2 Construct a treatment-self-healing coupled action space, introduce an irreversible damage screening mechanism for business capabilities, and obtain an effective action space. ;

[0036] S33.3, Effective motion space The output is sent to the decision generation and output module, serving as the range for reinforcement learning action selection.

[0037] As a further improvement to this technical solution, the reward and punishment function construction module constructs a multi-objective adaptive reward and punishment function based on a preset optimization objective. The process includes the following steps:

[0038] S34.1, the four core objectives are: success rate of integrated threat handling, business capability recovery rate, system self-healing success rate, and business interruption duration.

[0039] S34.2 Constructing a multi-objective adaptive reward and punishment function By allocating weights, optimization priorities are set for different threat scenarios, and a negative penalty mechanism is introduced.

[0040] S34.3, Using a multi-objective adaptive reward and punishment function As a basis for evaluating the value of reinforcement learning in the decision generation and output modules, it guides the iterative generation of safety response strategies and hierarchical self-healing action sequences.

[0041] As a further improvement to this technical solution, the verification and optimization module performs the following steps to verify and iteratively optimize the security handling strategy and the hierarchical self-healing action sequence based on the AI ​​agent's business capability baseline and security threat handling requirements:

[0042] S36.1 Establish a three-dimensional verification mechanism of "business security + handling effectiveness + resource rationality" and set verification standards. ;

[0043] S36.2, The effectiveness of the synchronous verification security handling strategy and the business security of the hierarchical self-healing action sequence were not achieved. In standard cases, gradient descent is used to adjust reinforcement learning parameters. ;

[0044] S36.3, Optimize the parameters The data is then sent back to the decision generation and output module to regenerate the safety handling strategy and the hierarchical self-healing action sequence until the verification is successful.

[0045] As a further improvement to this technical solution, the threat handling and self-healing execution unit includes a decision instruction receiving module, a threat handling execution module, a hierarchical self-healing execution module, a real-time business capability verification module, and a data acquisition and feedback module, wherein:

[0046] The decision instruction receiving module receives the verified security handling strategy and hierarchical self-healing action sequence output by the handling-self-healing decision generation unit.

[0047] The threat handling execution module performs security threat handling operations such as attack blocking and malicious code removal according to the security handling strategy;

[0048] The hierarchical self-healing execution module synchronously executes the AI ​​agent hierarchical self-healing operations of configuration restoration and node repair according to the hierarchical self-healing action sequence.

[0049] During the execution of handling and self-healing operations, the real-time business capability verification module compares the AI ​​agent's business capability baseline with the AI ​​agent's business capability status in real time.

[0050] The data acquisition and feedback module collects processing result data, self-healing result data, and business capability status data to form full-process execution feedback data, and outputs the full-process execution feedback data to the collaborative management and iterative optimization unit.

[0051] As a further improvement to this technical solution, the collaborative management and iterative optimization unit includes a dual-source data receiving module, a collaborative management module, a decision iterative optimization module, a knowledge graph updating module, and a data feedback module, wherein:

[0052] The dual-source data receiving module receives full-process execution feedback data output by the threat handling and self-healing execution unit, as well as security threat knowledge graph data output by the security threat knowledge graph construction unit;

[0053] The collaborative management module is based on security threat knowledge graph data and implements threat intelligence synchronization, coordinated action and unified scheduling of self-healing resources among multiple AI agents.

[0054] The decision iteration optimization module iteratively optimizes the decision generation process of the treatment-self-healing decision generation unit based on reinforcement learning, according to the feedback data of the entire process execution.

[0055] The knowledge graph update module updates the entity and relationship data of the security threat knowledge graph based on the feedback data of the entire process execution;

[0056] The data feedback module feeds back the updated security threat knowledge graph data to the security threat knowledge graph construction unit.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] 1. This invention extends the fusion application of knowledge graphs and reinforcement learning from the security threat reasoning and prediction stage to the entire process of security threat handling and self-healing operations of AI agents in industrial systems. It also constructs an action space that couples handling actions and self-healing actions, enabling the synchronous execution of threat handling operations and hierarchical self-healing operations of AI agents. Furthermore, by using the security threat knowledge graph as a priori constraint for reinforcement learning, it can also update the entities and relationships in the knowledge graph through execution feedback, forming a bidirectional fusion application system of knowledge graphs and reinforcement learning, and achieving deep linkage between the two in the security handling and self-healing stages.

[0059] 2. This invention addresses the actual security protection and operation needs of AI agents in industrial systems. It constructs a complete technical solution encompassing the entire process, from building a security threat knowledge graph, real-time security situation awareness and threat identification, to generating response and self-healing decisions, executing threat response and self-healing, and finally to collaborative management and iterative optimization. Furthermore, in the decision generation stage, it combines the threat state, vulnerability state, and degree of impairment of business capabilities of the AI ​​agent to construct a reinforcement learning state space, establishing a multi-dimensional verification mechanism to ensure business security. Simultaneously, it achieves threat intelligence synchronization among multiple AI agents, collaborative response actions, and unified scheduling of self-healing resources. This allows the fusion of knowledge graphs and reinforcement learning to be directly adapted to the actual security protection scenarios of AI agents and implemented in the actual execution of AI agent security threat response and self-healing. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the system framework of the present invention;

[0061] The meanings of the labels in the diagram are as follows:

[0062] 1. Security Threat Knowledge Graph Construction Unit; 11. Data Acquisition Module; 12. Data Association Processing Module; 13. Ontology Construction Module; 14. Graph Generation and Output Module;

[0063] 2. Security Situation Awareness and Threat Identification Unit; 21. Data Receiving and Parsing Module; 22. Entity Association Reasoning Module; 23. Situation Awareness and Threat Identification Module; 24. Data Generation and Output Module;

[0064] 3. Disposal-Self-Healing Decision Generation Unit; 31. Data Receiving Module; 32. State Space Construction Module; 33. Action Space Construction and Filtering Module; 34. Reward and Punishment Function Construction Module; 35. Decision Generation and Output Module; 36. Verification and Optimization Module;

[0065] 4. Threat handling and self-healing execution unit; 41. Decision instruction receiving module; 42. Threat handling execution module; 43. Hierarchical self-healing execution module; 44. Real-time business capability verification module; 45. Data acquisition and feedback module;

[0066] 5. Collaborative Management and Iterative Optimization Unit; 51. Dual-Source Data Receiving Module; 52. Collaborative Management Module; 53. Decision Iterative Optimization Module; 54. Knowledge Graph Update Module; 55. Data Feedback Module. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0068] like Figure 1 As shown, this embodiment provides an integrated system for handling and self-healing collaborative security threats using AI agents that combines knowledge graphs and reinforcement learning, including:

[0069] The security threat knowledge graph construction unit 1 collects AI agent operation data, security alarm data, vulnerability detection data, and AI agent business capability baseline data. Through named entity recognition, entity relationship alignment, and ontology construction technologies, it constructs a security threat knowledge graph containing core entities and their relationships, including agent nodes, business links, security threats, vulnerabilities, and business capability baselines. This security threat knowledge graph data is then output to the security situation awareness and threat identification unit 2. The security threat knowledge graph construction unit 1 includes a data acquisition module 11, a data association processing module 12, an ontology construction module 13, and a graph generation and output module 14, wherein:

[0070] In this embodiment, the data acquisition module 11 collects AI agent operation data, security alarm data, vulnerability detection data, and AI agent business capability baseline data. It completes the targeted acquisition and preliminary preprocessing of these data, providing unified and effective input data for the data association processing module 12. The specific implementation is as follows:

[0071] Lightweight distributed data acquisition agents are deployed on core AI agent nodes such as computing power nodes, inference nodes, collaboration nodes, and business execution nodes, as well as on the network security monitoring equipment, vulnerability scanning equipment, and business monitoring equipment that support the AI ​​agent. The data acquisition module 11 is configured with both push and pull acquisition modes. Push acquisition is used for AI agent operation data and security alarm data with high real-time requirements, while pull acquisition is used for periodically generated vulnerability detection data and AI agent business capability benchmark data.

[0072] Furthermore, the data acquisition module 11 collects core information from various dimensions according to data type. Among them, the AI ​​agent operation data collects data such as node hardware resource usage, process execution status, and inter-node collaborative interaction logs; the security alarm data collects data such as alarm type, alarm occurrence time, affected AI agent nodes / business links, and alarm level; the vulnerability detection data collects data such as vulnerability number, affected AI agent components, vulnerability risk level, and exploitation conditions; and the AI ​​agent business capability benchmark data collects benchmark indicators such as business response time, task completion rate, and node collaborative efficiency of the AI ​​agent under different business scenarios.

[0073] In addition, the data acquisition module 11 performs a unified format conversion on the acquired structured, semi-structured, and unstructured heterogeneous data, standardizing it into a unified format that can be parsed by the data association processing module 12. At the same time, it filters out abnormal data with missing core fields and invalid values, adds a unified identifier for each valid data, including the data source, acquisition time, and the AI ​​intelligent agent node to which it belongs, and transmits the data in batches to the data association processing module 12 after completing the preprocessing.

[0074] In this embodiment, the data association processing module 12 performs named entity recognition and entity relationship alignment on the data collected by the data acquisition module 11, extracts the five core entities required for the security threat knowledge graph: intelligent agent nodes, business links, security threats, vulnerabilities, and business capability baselines, establishes initial association relationships between entities, and provides a standardized set of entity and association relationship data for the ontology construction module 13. The specific implementation is as follows:

[0075] The data association processing module 12 builds a named entity recognition model adapted to the field of AI agent security. Based on the exclusive corpus of AI agent security, the model is customized and fine-tuned. The preprocessed data transmitted by the data acquisition module 11 is input into the model to complete the accurate identification and extraction of five core entities: agent nodes, business links, security threats, vulnerabilities, and business capability baselines.

[0076] Meanwhile, the data association processing module 12 performs entity normalization processing on the five identified core entities, unifies the naming and description rules of each entity, and eliminates the problem of heterogeneous representation of the same entity.

[0077] Subsequently, the data association processing module 12 constructs an entity relationship matching rule base based on the business operation logic and security protection characteristics of the AI ​​intelligent agent. By calculating the similarity of entity attributes and the similarity of context scenarios, it matches and associates entities in cross-source data to complete the entity relationship alignment operation. At the same time, it performs deduplication, completion, and cleaning on the aligned entity relationships, and removes invalid and conflicting association relationships. Finally, it forms a standardized data set containing five core entities, entity attributes, and effective association relationships between entities, and transmits this data set to the ontology construction module 13.

[0078] In this embodiment, the ontology construction module 13 constructs core entities and their relationships, including intelligent agent nodes, business links, security threats, vulnerabilities, and business capability baselines, based on the processing results of the data association processing module 12. It then constructs a domain ontology model containing these core entities and their relationships using ontology construction technology. This provides a structured framework for the graph generation and output module 14 to construct the security threat knowledge graph. The specific implementation is as follows:

[0079] Based on the standardized data set output by the data association processing module 12, the ontology construction module 13 determines the core scope of ontology construction as five core entities in the scenario of AI intelligent agent security threat handling and self-healing; it extracts sub-class concepts of the five core entities from the standardized data set, constructs a three-level hierarchical concept system of "top-level core entity - middle-level sub-class concept - bottom-level specific instance", and clarifies the hierarchical classification relationship between each entity;

[0080] Meanwhile, based on the entity association relationship output by the data association processing module 12, and combined with the business and security rules in the field of AI intelligent agent security, the semantic association relationship between the five core entities and sub-categories is defined, and the association types of one-to-one, one-to-many, and many-to-many are clarified.

[0081] Furthermore, dedicated attribute fields are configured for the five core entities and each sub-category concept. The attribute fields are consistent with the collection dimensions of the data acquisition module 11 and the entity attributes of the data association processing module 12, thus improving the feature description of the entities. The ontology construction module 13 performs consistency and integrity verification on the completed ontology model, corrects problems such as concept hierarchy conflicts and entity relationship contradictions, and finally forms an ontology model containing core entities and related relationships of intelligent agent nodes, business links, security threats, vulnerabilities, business capability baselines, and transmits the model to the graph generation output module 14.

[0082] In this embodiment, the graph generation and output module 14 generates a security threat knowledge graph based on the construction results of the ontology construction module 13, and outputs the security threat knowledge graph data to the security situation awareness and threat identification unit 2, thus completing all the core operations of the security threat knowledge graph construction unit 1. The specific implementation is as follows:

[0083] The graph generation and output module 14 uses a graph database adapted for graph structure data storage and query as a carrier. It maps the five core entities in the ontology model transmitted by the ontology construction module 13—intelligent agent nodes, business links, security threats, vulnerabilities, and business capability baselines—to nodes in the graph database, maps the relationships between entities to edges connecting nodes in the graph database, and maps the attributes of entities and relationships to the attributes of nodes and edges, respectively, thus completing the graph structure construction of the security threat knowledge graph.

[0084] Meanwhile, the graph generation and output module 14 performs integrity verification on the completed security threat knowledge graph, checks the coverage of the five core entities and the integrity of the relationships between key entities, and ensures that there are no missing core entity nodes or omissions of key business and security relationships.

[0085] Furthermore, after verification, the graph generation and output module 14 persistently stores the security threat knowledge graph in the graph database. At the same time, it builds a standardized data output interface to encapsulate the security threat knowledge graph data in a unified format. Through this interface, the complete security threat knowledge graph data is output to the security situation awareness and threat identification unit 2, providing structured and accurate knowledge graph support for the subsequent operations of the security situation awareness and threat identification unit 2.

[0086] It should be added that the security threat knowledge graph construction unit 1 works collaboratively in a serial process of "data acquisition → data association processing → ontology construction → graph generation and output". The modules realize unidirectional and orderly data transmission, and finally output the security threat knowledge graph data to the security situation awareness and threat identification unit 2 through the graph generation and output module 14. The security threat knowledge graph construction unit 1 reserves a standardized data interaction interface, which can receive the full-process execution feedback data from the collaborative management and iterative optimization unit 5. Based on this data, the entities and relationships of the security threat knowledge graph are dynamically updated and supplemented, continuously optimizing the accuracy and completeness of the security threat knowledge graph, and providing continuous and effective knowledge support for the entire process operation of the system.

[0087] The security situation awareness and threat identification unit 2 receives security threat knowledge graph data output by the security threat knowledge graph construction unit 1, performs entity relationship reasoning based on the entity relationships in the security threat knowledge graph data, completes real-time security situation awareness and threat identification for the AI ​​agent, and generates and outputs standardized threat relationship data, AI agent business capability baseline data, and business link and security status relationship data, which are synchronously transmitted to the handling-self-healing decision generation unit 3. The security situation awareness and threat identification unit 2 includes a data receiving and parsing module 21, an entity relationship reasoning module 22, a situation awareness and threat identification module 23, and a data generation and output module 24, wherein:

[0088] In this embodiment, the data receiving and parsing module 21 receives the security threat knowledge graph data output by the security threat knowledge graph construction unit 1, performs format parsing and validity verification on the security threat knowledge graph data, filters invalid data, and transmits the compliant structured graph data to the entity association reasoning module 22, providing a parsable and highly reliable data source for subsequent reasoning operations. The specific implementation is as follows:

[0089] The data receiving and parsing module 21 receives the security threat knowledge graph data output by the graph generation and output module 14 of the security threat knowledge graph construction unit 1 through a standardized RESTful API interface, and supports receiving two data formats: JSON-LD standard format and Cypher query result set.

[0090] Simultaneously, the received security threat knowledge graph data is first parsed to convert graph data of different formats into a structured graph data format that can be uniformly parsed by each module of the security situation awareness and threat identification unit 2, and extract information on intelligent agent nodes, business links, security threats, vulnerabilities, core entities of business capability baseline and entity relationship information from the data.

[0091] Subsequently, a multi-dimensional validity check is performed. The check dimensions include data integrity (checking whether the five core entities and key relationships are missing), format compliance (checking whether data fields match the preset specifications and whether the identifiers are unique), and data timeliness (checking whether the timestamp of the graph data is the latest update). For invalid data that fails the check, the data receiving and parsing module 21 sends data anomaly information to the graph generation and output module 14 and requests retransmission. For compliant structured graph data that passes the check, it is temporarily cached locally and then transmitted to the entity association reasoning module 22.

[0092] In this embodiment, the entity association reasoning module 22 performs entity association reasoning based on the entity associations in the security threat knowledge graph data parsed by the data receiving and parsing module 21, and outputs the entity association reasoning results. This provides the core reasoning basis for the AI ​​agent's real-time security situation awareness and threat identification. The specific implementation is as follows:

[0093] The entity association reasoning module 22 retrieves the compliant structured graph data transmitted by the data receiving and parsing module 21, and builds a rule-based deductive reasoning engine based on the core entity association rules preset in the security threat knowledge graph, namely "security threat-exploitation-vulnerability", "vulnerability-existence-agent node", "agent node-composition-business link", and "business link-correspondence-business capability baseline".

[0094] Furthermore, the inference engine performs a full traversal of entities and relationships in the graph data to uncover potential security connections between entities, including the impact links of security threats exploiting vulnerabilities on agent nodes, the business links affected by the affected agent nodes, and the potential impact of damaged business links on the corresponding business capability baseline. At the same time, the inference results are supplemented by combining the real-time operating characteristics of AI agents, and the impact range and propagation probability of the inferred connection links are marked.

[0095] After the reasoning is completed, the entity association reasoning module 22 will organize the reasoning results, which include entity association links, potential security associations, affected entity ranges, and threat propagation paths, into a structured form, encapsulate them in a preset format, and transmit them to the situation awareness and threat identification module 23.

[0096] In this embodiment, the situational awareness and threat identification module 23 receives the entity association reasoning results output by the entity association reasoning module 22, completes real-time security situational awareness and threat identification of the AI ​​agent, clarifies the current overall security status of the AI ​​agent, specific threat information, and the impact of the security status on business, and forms comprehensive data containing situational awareness and threat identification results, providing core data support for the data generation and output module 24. The specific implementation is as follows:

[0097] The situational awareness and threat identification module 23 receives the structured entity association reasoning results transmitted by the entity association reasoning module 22. It first performs real-time security situational awareness operations on the AI ​​agent, and constructs a situational awareness system from three dimensions: agent nodes, business links, and overall business capabilities. It analyzes in real time the security status (normal / vulnerable / affected by threats) of each agent node, the access status (normal / partial node damage / link interruption) of each business link, and the baseline matching status of the business capabilities corresponding to each business link (normal / deviation / serious deviation) in the reasoning results. It dynamically draws a real-time security situational profile of the AI ​​agent, and realizes real-time perception and visualization of the overall security situation of the AI ​​agent.

[0098] Furthermore, based on the situational awareness results, precise threat identification operations are carried out. The specific type, triggering conditions, associated vulnerability number, and affected intelligent agent nodes / business links of the security threat entities are extracted from the reasoning results. The identified threats are labeled according to the preset security threat level classification standards (such as high-risk, medium-risk, and low-risk). At the same time, it is clear whether the threat has caused damage to the business capabilities of the AI ​​intelligent agent and the specific extent of the damage.

[0099] In addition, after completing perception and identification, the situational awareness and threat identification module 23 will transmit comprehensive data, including multi-dimensional security situational information of the AI ​​agent, threat type / level / scope of impact, and the impact on business capabilities, to the data generation and output module 24.

[0100] In this embodiment, the data generation and output module 24 generates and outputs standardized threat correlation data, AI agent business capability baseline data, and business link and security status correlation data based on the identification results of the situational awareness and threat identification module 23. This data is then synchronously transmitted to the handling-self-healing decision generation unit 3, providing standardized, structured, and targeted input data for the generation of downstream handling and self-healing decisions. The specific implementation is as follows:

[0101] The data generation and output module 24 receives the comprehensive data transmitted by the situational awareness and threat identification module 23, and constructs three types of core output data according to preset data standards:

[0102] Firstly, to standardize threat association data, core information such as threat type, threat level, associated vulnerability, affected agent nodes / business links, and threat propagation path is integrated, and data fields and identification rules are unified;

[0103] Secondly, for the baseline data of AI agent business capabilities, extract the baseline indicators of business capabilities corresponding to each business link in the knowledge graph, and combine them with the real-time situational awareness results to mark the actual operating value of each business capability baseline of the current AI agent, the degree of deviation from the benchmark value, and the reasons for the deviation.

[0104] Third, establish a one-to-one mapping relationship between each business link and its corresponding security status, and mark information such as the degree of threat impact, core damaged nodes, and link throughput efficiency for each business link;

[0105] After the three types of data are generated, the data generation and output module 24 standardizes them into JSON format, adds fields such as unique identifier, generation timestamp, and AI agent identifier to each data, and transmits the three types of data synchronously and in real time to the disposal-self-healing decision generation unit 3 through an encrypted transmission interface. At the same time, the three types of standardized data generated are stored locally and a copy of the data is retained for subsequent system operation traceability and iterative optimization.

[0106] It should be added that the security situation awareness and threat identification unit 2 maintains real-time data interaction with the security threat knowledge graph construction unit 1, and synchronously receives the updated graph data from the security threat knowledge graph construction unit 1 to achieve dynamic updates of perception and identification results. At the same time, the security situation awareness and threat identification unit 2 reserves a data receiving interface to receive full-process execution feedback data from the collaborative management and iterative optimization unit 5. Based on this data, the entity association reasoning rules and situation awareness dimensions are optimized to continuously improve the accuracy of the AI ​​intelligent agent's security situation awareness and the timeliness of threat identification, providing continuous and reliable perception support for the entire system's handling and self-healing decision-making.

[0107] The Disposal-Self-Healing Decision Generation Unit 3 receives standardized threat correlation data, AI agent business capability baseline data, and business link and security status correlation data output by the Security Situation Awareness and Threat Identification Unit 2. Using a security threat knowledge graph as a prior constraint for reinforcement learning, it constructs a reinforcement learning state space based on threat status, vulnerability status, and the degree of business capability impairment. Based on the business link and security status correlation rules, it constructs an action space coupling disposal actions and self-healing actions, eliminating actions that would cause irreversible damage to the AI ​​agent's business capabilities. The decision is based on threat disposal success rate, business capability recovery rate, and system self-healing... A multi-objective reward and punishment function is constructed with success rate and business interruption duration as optimization goals. Reinforcement learning is used to generate security handling strategies and hierarchical self-healing action sequences adapted to the current threat. The action sequences are validated against the business capability baseline; if a preset validation threshold is not met, iterative optimization is performed. The validated security handling strategies and self-healing action sequences are output to the threat handling and self-healing execution unit 4. The handling-self-healing decision generation unit 3 includes a data receiving module 31, a state space construction module 32, an action space construction and filtering module 33, a reward and punishment function construction module 34, a decision generation and output module 35, and a validation and optimization module 36, wherein:

[0108] In this embodiment, the data receiving module 31 receives standardized threat correlation data, AI agent business capability baseline data, and business link and security status correlation data output by the security situation awareness and threat identification unit 2, and completes data format unification, field alignment, and validity filtering to provide a highly reliable input data source for the subsequent reinforcement learning decision-making process. The specific implementation is as follows:

[0109] The data receiving module 31 is equipped with an encrypted bidirectional communication interface, which supports batch / real-time transmission mode in JSON format. It receives standardized threat-related data, AI intelligent agent business capability baseline data, and business link and security status related data transmitted by the data generation and output module 24.

[0110] First, perform format compliance verification, check the completeness of the core fields of the three types of data (such as threat-related data must include threat ID, level, and associated vulnerability ID, and business baseline data must include link ID, baseline value, and current value), mark abnormal data with missing fields or incorrect formats, and report them to the security situation awareness and threat identification unit 2;

[0111] Secondly, align the fields of compliant data and unify the naming rules and data types of common identifier fields such as "intelligent agent node ID, business link ID, and timestamp" (e.g., unify node ID to "cluster-node" format and timestamp to millisecond level).

[0112] Finally, a validity screening is performed to remove invalid data that exceeds the real-time decision window (default 5 minutes) or whose associated entities are not included in the security threat knowledge graph. The screened compliant data is then structured and integrated into a decision input dataset according to the "threat-vulnerability-business" dimension, cached locally, and then transmitted to the state space construction module 32.

[0113] Furthermore, for the three extreme abnormal scenarios in this embodiment, the following fallback strategies are set: First, when the data input is empty or there is no valid decision data, a preset combination of basic protection actions is output; second, when there is no matching entity subset in the knowledge graph, a threat handling-self-healing action combination commonly used in this field is invoked; third, the effective action space after filtering... If the data is empty, an alarm is immediately triggered and the decision-making process is paused. At the same time, the data is fed back to the security situation awareness and threat identification unit 2 for re-collection.

[0114] In this embodiment, the state space construction module 32 constructs the reinforcement learning state space using the security threat knowledge graph as a prior constraint for reinforcement learning. The state space construction module 32 uses the security threat knowledge graph as a prior constraint for reinforcement learning to construct the reinforcement learning state space. The process includes the following steps:

[0115] S32.1. Combining the entity relationships in the security threat knowledge graph, we determine the three core state dimensions of threat, vulnerability, and business impairment. We extract the entity relationships of "security threat - exploitation - vulnerability", "vulnerability - exists in - agent node", "agent node - composes - business link", and "business link - corresponds to - business capability baseline" as prior constraints to clarify the reinforcement learning state space. The three core dimensions are the threat state dimension. Vulnerability State Dimension Business damage dimension The three dimensions correspond one-to-one with the core entities in the security threat knowledge graph, ensuring a strong correlation between the state space construction and the core business scenarios of AI agents' security threat handling and self-healing. Specifically, the core attributes of the knowledge graph strongly coupled with the state space construction in this embodiment include the association weight attribute of the "security threat-exploitation-vulnerability" relationship: assigned values ​​according to threat level, with high-risk / medium-risk / low-risk corresponding to 1 / 0.6 / 0.2 respectively, directly constraining the threat state dimension. Quantitative calculation; Business carrying weight attribute of the "Agent Node-Composition-Business Link" relationship: marks the business priority of the node in the link, used to distinguish core / non-core business nodes; Baseline threshold attribute of the "Business Link-Correspondence-Business Capability Baseline" relationship: records the normal baseline range of business capabilities, as the dimension of business impairment. A benchmark for deviation rate calculation is provided. The above attributes provide direct prior constraints for the quantitative calculation of the three state dimensions, ensuring that the state space is deeply bound to the business scenario of this embodiment.

[0116] The state space construction module 32 first retrieves entity relationship data from the security threat knowledge graph. It then uses Cypher query statements to filter out a subset of entities that match the output data of the current security situation awareness and threat identification unit 2. The core matching condition is to match the "security threat-exploitation-vulnerability" relationship entity corresponding to the current security threat ID, further associate the "vulnerability-existence-agent node" relationship entity corresponding to the vulnerability entity, then associate the "agent node-composition-business link" relationship entity corresponding to the agent node, and finally associate the "business link-correspondence-business capability baseline" relationship entity corresponding to the business link. This process filters out a subset of entities that are only related to the current threat scenario, avoiding irrelevant entities from interfering with the state space construction and ensuring the accuracy of the quantification boundary.

[0117] Based on the association logic of this entity subset, the quantification boundaries of the three core dimensions are determined: threat status dimension. Quantify the level and scope of impact of security threats, and the vulnerability status dimension. Quantify the risk level and probability of exploitation of vulnerabilities, and the dimensions of business damage. The deviation of the AI ​​agent's business capabilities from the baseline value is quantified by three dimensions that together constitute the reinforcement learning state space. The core framework.

[0118] S32.2 Constructing a reinforcement learning state space based on three core state dimensions. The three core state dimensions are quantized and normalized, and then integrated into a reinforcement learning state space in the form of a three-dimensional vector. The core algorithm employs a knowledge graph-constrained dimensional quantization normalization algorithm. The improvement lies in assigning contextualized weights to each dimension's quantization process through prior associations within the security threat knowledge graph. This addresses the issues of dimensional quantization being detached from business scenarios and exhibiting poor generalization in traditional reinforcement learning state space construction. The calculation logic and formulas are as follows:

[0119] First, construct the reinforcement learning state space. The core expression is:

[0120] ;

[0121] in:

[0122] Representing the reinforcement learning state space, it is a three-dimensional column vector and is the core state input of the reinforcement learning algorithm in the disposition-self-healing decision generation unit 3;

[0123] The quantified value representing the threat status dimension is a normalized value in the range of 0-1;

[0124] The quantified value representing the vulnerability state dimension is a normalized value in the range of 0-1;

[0125] This represents the quantified value of the dimension of business impairment, which is a normalized value in the range of 0-1.

[0126] Furthermore, quantitative calculations were performed on the three core dimensions respectively:

[0127] Threat Status Dimension Quantization calculation:

[0128] Based on the relationship between "security threat - impact - agent node" in the security threat knowledge graph, and combined with the threat level and the proportion of impact nodes, the formula is as follows:

[0129] ;

[0130] in:

[0131] The quantification value representing the level of security threats is based on the threat level definition in the security threat knowledge graph, such as 1 for high-risk threats, 0.6 for medium-risk threats, and 0.2 for low-risk threats;

[0132] The weight of the node affected by the threat is the ratio of the number of agent nodes affected by the threat to the total number of AI agent nodes, and the value range is (0,1].

[0133] express The theoretical maximum value is 1 (when and (Time), used to normalize the calculation results to the 0-1 interval.

[0134] Vulnerability State Dimension Quantization calculation:

[0135] Based on the relationship between "security threat-exploitation-vulnerability" in the security threat knowledge graph, and combined with the vulnerability risk level and the probability of exploitation, the formula is as follows:

[0136] ;

[0137] in:

[0138] The risk level quantification value representing vulnerability is based on the vulnerability level definition in the security threat knowledge graph, with high-risk vulnerability taking 1, medium-risk vulnerability taking 0.6, and low-risk vulnerability taking 0.2.

[0139] This represents the probability that a vulnerability will be exploited by a current security threat. It is calculated based on historical correlation data in the security threat knowledge graph and has a value range of (0,1].

[0140] express The theoretical maximum value is 1 (when =1 and (Time), used to normalize the calculation results to the 0-1 interval.

[0141] Business damage dimensions Quantization calculation:

[0142] Based on the relationship between "business link - corresponding - business capability baseline" in the security threat knowledge graph, the deviation rate between the current business capability and the baseline value is calculated using the following formula:

[0143] ;

[0144] in:

[0145] This represents the average current business capability of the AI ​​agent, which is the normalized average of core business indicators such as business response time, task completion rate, and computing power utilization. The weights of each indicator are assigned based on the business link priority definition in the security threat knowledge graph; and the average business capability... The weights of the indicators follow the priority mapping logic of the business links. The task completion rate weight of the core business links accounts for ≥50%, and the weights of each indicator of the non-core business links are allocated according to the business value gradient. Those skilled in the art can complete the specific weight assignment based on this logic.

[0146] This represents the baseline value of the business capabilities of the corresponding business links preset in the security threat knowledge graph, which is the average value of the core business indicators under normal operating conditions of the AI ​​agent.

[0147] Finally, the quantized calculations Substituting the core expressions of the state space, we obtain the complete reinforcement learning state space. This completes the core construction work of this step.

[0148] S32.3, Strengthen the learning state space The output is sent to the decision generation and output module 35 as the state input for reinforcement learning, and is used to construct the reinforcement learning state space. The format is encapsulated and validity is validated to ensure that it can be directly parsed and used by the reinforcement learning algorithm in the decision generation and output module 35. The specific implementation is as follows:

[0149] State space construction module 32 first constructs the state space. Perform validity checks; the validation logic is confirmation. The values ​​are all within the range of 0-1, and there are no null or outlier values. If the validation fails, the process will backtrack to step S32.2 to recalculate the dimension quantization; if the validation passes, the state space will be... According to the input format requirements of the decision generation and output module 35, the state input data is encapsulated in tensor form, with tensor dimensions of [1,3] (1 batch, 3 state dimensions).

[0150] Subsequently, the state space construction module 32 transmits the encapsulated reinforcement learning state space through its internal standardized data transmission interface. The output is directed to the decision generation and output module 35, while the state space is retained locally. The computation logs (including quantified values ​​for each dimension, computation timestamps, and knowledge graph-related data identifiers) are used for subsequent system iteration optimization and data traceability, reinforcing the learning state space. As the core state input of the deep Q-network algorithm in the decision generation and output module 35, it provides the basic state basis for the generation of safety handling strategies and hierarchical self-healing action sequences.

[0151] In this embodiment, the action space construction and filtering module 33 constructs and filters the action space coupled with handling actions and self-healing actions based on the association rules between business links and security status. The process by which the action space construction and filtering module 33 constructs and filters the action space coupled with handling actions and self-healing actions based on the association rules between business links and security status includes the following steps:

[0152] S33.1 Establish the linkage and coupling logic between handling actions and self-healing actions, clarify the timing and correlation of action combinations, provide a rule basis for the subsequent construction of the coupled action space, and avoid business risks caused by the disconnection and timing disorder of handling and self-healing actions. The specific implementation is as follows:

[0153] The action space construction and filtering module 33 first extracts two core action sets from the security threat knowledge graph, as follows:

[0154] Set of handling actions This includes security actions such as attack blocking, malicious code removal, temporary vulnerability patching, and traffic isolation. Each action has a unique identifier. ;

[0155] Self-healing action collection This includes system self-healing actions such as configuration restoration, node repair, link reconstruction, resource scheduling, and service rollback. Each action corresponds to a unique identifier. .

[0156] Furthermore, based on the association rules between business links and security status, a linkage and coupling logic is established, as follows:

[0157] Timing constraint rules:

[0158] Define the execution sequence of the action combination (positive integer, , ),in To prioritize the execution of actions, The next priority action is executed, and so on, adapting to the action sequence length of the AI ​​agent's real-time decision-making. The core timing rule is: handling actions take precedence over self-healing actions (such as attack blocking first, then node repair), and link-level self-healing actions take precedence over node-level self-healing actions (such as reconstructing the business link first, then repairing the damaged node).

[0159] Association constraint rules:

[0160] Based on the "threat-business link-impaired node" relationship in the security threat knowledge graph, clarify the relationship between action combinations, for example:

[0161] If the security threat is "link interruption", then the action "traffic isolation" must be combined with the self-healing action "link reconstruction", in sequence. Implement "traffic isolation" Perform "link reconstruction";

[0162] If the security threat is "malicious code infection", then the action "malicious code removal" must be combined with the self-healing action "configuration restoration", in sequence. Perform "malicious code removal". Perform "Configuration Restore".

[0163] It should be added that the core matching basis for threat type and handling-self-healing action combination is the "impact carrier" attribute of the threat in the knowledge graph: threats affecting business links are matched with traffic isolation and link reconstruction action combinations; threats affecting agent nodes are matched with attack blocking and node configuration restoration action combinations; threats affecting code / processes are matched with malicious code removal and business rollback action combinations. Those skilled in the art can complete the action matching for all threat types based on this basis.

[0164] In addition, the action space construction and filtering module 33 organizes the above-mentioned linkage and coupling logic into a rule base and stores it locally, providing a constraint basis for the subsequent construction of the coupled action space.

[0165] S33.2 Construct a treatment-self-healing coupled action space, introduce an irreversible damage screening mechanism for business capabilities, and obtain an effective action space. This involves eliminating action combinations that could cause irreversible damage to the AI ​​agent's business capabilities, providing a safe and feasible range of action choices for reinforcement learning, and preventing decisions from causing irreversible business damage. The specific implementation is as follows:

[0166] First, construct the initial treatment-self-healing coupled action space. :

[0167] The action space construction and filtering module 33 constructs an initial treatment-self-healing coupled action space based on linkage coupling logic. The formula is:

[0168] ;

[0169] in:

[0170] The initial handling-self-healing coupled action space is a set of action combinations that satisfy the linkage and coupling logic.

[0171] This represents a time sequence set, with values ​​ranging from 1 to 5, which is adapted to the length of action sequences for real-time decision-making by AI agents.

[0172] Then, an irreversible damage screening mechanism for business capabilities is introduced:

[0173] The Action Space Construction and Filtering Module 33 introduces an irreversible damage filtering mechanism for business capabilities, defining the rules for determining irreversible damage to business capabilities: Action Combinations After execution, the predicted deviation rate of the AI ​​agent's business capabilities. And the duration exceeds If so, it is determined to have caused irreversible damage to business capabilities.

[0174] This represents the predicted deviation rate of business capabilities after the execution of a combination of actions. It is obtained by fitting historical action-business impact data from a security threat knowledge graph, and the calculation formula is as follows:

[0175] ;

[0176] in:

[0177] The predicted average of business capabilities after the execution of the action combination;

[0178] The baseline values ​​for business capabilities preset in the security threat knowledge graph;

[0179] This represents the threshold for determining irreversible damage to business capabilities, and can be set to 0.8 (it can be dynamically adjusted according to the importance of the business; for core business scenarios, it can be lowered to 0.6).

[0180] This represents the threshold for the duration of irreversible damage to business capabilities, with a value of 5 minutes (which can be dynamically adjusted based on business recovery capabilities).

[0181] Finally, the effective action space is obtained through filtering. :

[0182] The action space construction and filtering module 33, based on the irreversible damage determination rule, performs initial coupled action space construction and filtering. By filtering and eliminating action combinations that cause irreversible damage to business capabilities, an effective action space can be obtained. The formula is:

[0183] ;

[0184] in:

[0185] The effective action space is the set of safe and feasible action combinations after eliminating action combinations that result in irreversible damage.

[0186] Indicates a combination of actions Predicted deviation rate of business capabilities after execution It was obtained by fitting historical data from a security threat knowledge graph.

[0187] Understandably, the business capability irreversible damage screening mechanism introduced in this step solves the problem of "action combinations may cause irreversible damage to business" in the traditional reinforcement learning action space. By fitting and predicting the business impact through historical data of the knowledge graph, harmful actions are eliminated in advance, improving the security and feasibility of the action space.

[0188] S33.3, Effective motion space The output is sent to the decision generation and output module 35, which serves as the range for reinforcement learning action selection. This ensures that the reinforcement learning algorithm generates decisions only from safe and feasible action combinations, improving the safety and feasibility of the decisions. At the same time, it provides clear action selection boundaries for subsequent decision generation. The specific implementation is as follows:

[0189] The motion space construction and filtering module 33 first examines the effective motion space. Perform validity checks:

[0190] Check the timing of action combinations Is it in Within the scope, eliminate action combinations with abnormal timing;

[0191] Check whether the relationships between action combinations conform to the linkage and coupling logic, and eliminate action combinations with mismatched relationships;

[0192] Predicted deviation rate of business capability for inspection action combinations Does it meet the requirements? Action combinations that fail the irreversible damage screening are eliminated.

[0193] After successful verification, the action space construction and filtering module 33 will select the effective action space. According to the sequence of action combinations The actions are sorted and encapsulated into a list of action indices that can be parsed by reinforcement learning algorithms (each action combination corresponds to a unique index). This list is then output to the decision generation and output module 35 via an internal standardized data transmission interface. Simultaneously, an action space log (containing action combination identifiers, time series, predicted business impact values, and filtering rule identifiers) is retained locally for subsequent system iteration optimization and data traceability. Effective Action Space As the action selection range of the deep Q-network algorithm in the decision generation and output module 35, it ensures that reinforcement learning decisions are generated only from safe and feasible action combinations.

[0194] It should be added that the core determination rules for the subject of action execution synchronously marked in the above action index list are as follows: Actions such as attack blocking, malicious code removal, and temporary vulnerability blocking are executed at the node where the threat enters the site or the intelligent agent node directly affected by the threat; Self-healing actions such as link reconstruction, resource scheduling, and business rollback are executed at the core forwarding node of the corresponding business link. Those skilled in the art can determine the specific subject of execution based on these rules.

[0195] Furthermore, the core determination rule for action execution nodes is as follows: using the "threat tracing node" marked in the security threat knowledge graph as the core benchmark, the execution node for handling actions is extended to the upstream and downstream adjacent nodes of the threat tracing node (the extension range does not exceed 2 hops), avoiding indiscriminate expansion of the execution scope and resulting system performance loss; the execution node for self-healing actions is strictly limited to the effective node range of the corresponding business link (the effective node is the node that carries the core function of the business, marked by the core attribute "business link - containment - node" in the knowledge graph), and only nodes with a current resource utilization rate of less than 70% are selected for execution, ensuring that the execution of self-healing actions does not cause new resource bottlenecks. Based on this rule, those skilled in the art can accurately determine the specific execution node for each action combination by combining the actual threat scenario and node resource status.

[0196] In this embodiment, the reward and punishment function construction module 34 constructs a multi-objective adaptive reward and punishment function based on a preset optimization objective. The reward and punishment function construction module 34 constructs a multi-objective adaptive reward and punishment function based on preset optimization goals. The process includes the following steps:

[0197] S34.1, with four core objectives—threat handling success rate, business capability recovery rate, system self-healing success rate, and business interruption duration—as the basis for a multi-objective adaptive reward and punishment function. Provide a quantitative basis to ensure that value assessment covers four core dimensions: security handling, business recovery, system self-healing, and business continuity, avoiding decision-making imbalances caused by optimizing a single objective. Specifically, this is achieved as follows:

[0198] The reward and punishment function construction module 34 first extracts quantitative indicators for four core objectives from the output data of the security threat knowledge graph and the security situation awareness and threat identification unit 2, specifically including:

[0199] Threat handling success rate :

[0200] The effectiveness of quantifiable safety response actions is calculated using the following formula:

[0201] ;

[0202] in:

[0203] The number of security threats that have been successfully dealt with can be obtained from the "Threat Handling Status" field of the security threat knowledge graph;

[0204] This indicates the total number of security threats in the current scenario, which can be extracted from the output data of the security situation awareness and threat identification unit 2.

[0205] The value ranges from [0,1], and the larger the value, the better the threat handling effect.

[0206] Business capability recovery rate :

[0207] The formula for calculating the extent to which business capabilities have recovered from a damaged state to the baseline is as follows:

[0208] ;

[0209] in:

[0210] This represents the average current business capability, which can be extracted from the output data of the security situation awareness and threat identification unit 2.

[0211] This represents the average impairment of business capabilities after a threat occurs, which can be extracted from historical data in the security threat knowledge graph.

[0212] This represents the baseline value of business capabilities, which can be read from the security threat knowledge graph;

[0213] The value ranges from [0,1], and the larger the value, the better the business recovery effect.

[0214] System self-healing success rate :

[0215] The effectiveness of the quantification system's self-healing action is calculated using the following formula:

[0216] ;

[0217] in:

[0218] The number of successfully executed self-healing actions can be obtained from the "Self-healing Execution Status" field of the security threat knowledge graph;

[0219] This represents the total number of self-healing actions in the current scene, which can be derived from the effective action space. Extract the total number of independent self-healing actions;

[0220] The value ranges from [0,1], and the larger the value, the better the system's self-healing effect.

[0221] Service interruption duration :

[0222] The duration of the quantified business interruption, in minutes, can be extracted from the output data of the security situation awareness and threat identification unit 2. The value range is [0, +∞), and the larger the value, the worse the business continuity.

[0223] Furthermore, the reward and punishment function construction module 34 normalizes the quantitative indicators of the four core objectives mentioned above, ensuring that the value range of each indicator is uniformly [0,1], among which the business interruption duration... The normalization formula is:

[0224] ;

[0225] in The preset maximum service interruption duration is 30 minutes, used to... Normalize to the [0,1] interval.

[0226] S34.2 Constructing a multi-objective adaptive reward and punishment function By allocating weights, the optimization focus for different threat scenarios is achieved, and a negative penalty mechanism is introduced to provide accurate value assessment basis for reinforcement learning, avoiding business risks caused by neglecting business continuity or excessively pursuing processing efficiency in decision-making. The specific implementation is as follows: The reward and penalty function construction module 34 constructs a multi-objective adaptive reward and penalty function based on the four normalized core objectives. The formula is:

[0227] ;

[0228] in:

[0229] This represents the value of the multi-objective adaptive reward and punishment function, which ranges from [−1, 1]. A larger value indicates a better overall effect of the action combination, while a negative value indicates that the negative impact outweighs the positive benefit.

[0230] Weighting coefficients, satisfying This is used to optimize different threat scenarios.

[0231] Weight allocation mechanism (adaptive optimization emphasis):

[0232] The reward and punishment function construction module 34 dynamically adjusts the weight coefficients based on the threat level and business link priority in the security threat knowledge graph. To achieve optimization focus for different threat scenarios:

[0233] High-risk threat scenarios: increase (Threat handling success rate weighting), for example Prioritize ensuring the effectiveness of threat response;

[0234] Core business affected scenarios: increase (Business capability recovery rate weight) and (Weight of business interruption duration), for example Prioritize ensuring business continuity and recovery;

[0235] System self-healing requirement scenario: increase (System self-healing success rate weight), for example Prioritize ensuring the system's self-healing effect.

[0236] Negative punishment mechanism:

[0237] Module 34 of the reward and punishment function construction module introduces a negative penalty mechanism, which applies when the service is interrupted for a certain period of time. Exceeding the preset threshold (When the value is 10 minutes), the reward / penalty function value is... Additional penalties will be imposed, and the penalty formula is as follows:

[0238] ;

[0239] in:

[0240] This represents the penalty coefficient, with a value of 0.5, used to control the intensity of the penalty.

[0241] This represents the normalized service interruption duration penalty threshold, calculated using the following formula: ; This indicates the maximum preset service interruption duration, in minutes, used to specify the service interruption duration. Normalization to Interval.

[0242] Understandably, this step achieves multi-objective adaptive optimization through weight allocation, solving the problem of single-objective optimization in traditional reward and punishment functions; it introduces a negative penalty mechanism to impose additional penalties on decisions with excessively long business interruption times, strengthening the guarantee of business continuity and improving the scenario adaptability and rationality of reward and punishment functions.

[0243] S34.3, Using a multi-objective adaptive reward and punishment function As the basis for evaluating the value of reinforcement learning in the decision generation and output module 35, it guides the iterative generation of security handling strategies and hierarchical self-healing action sequences, ensuring that reinforcement learning decisions iterate within a multi-objective optimization framework, taking into account security handling, business recovery, system self-healing, and business continuity. The specific implementation is as follows:

[0244] The reward and punishment function construction module 34 will construct the multi-objective adaptive reward and punishment function. Through the internal standardized data transmission interface, the data is directed to the decision generation and output module 35, while the reward and punishment function log (including weight coefficient, punishment threshold, and target indicator value) is retained locally for subsequent system iteration optimization and data traceability.

[0245] Meanwhile, the multi-objective adaptive reward and punishment function As the basis for evaluating the value of the deep Q-network algorithm in the decision generation and output module 35, the reward and penalty value of the action combination is calculated after each action selection. This guides the parameter updates and decision iterations of deep Q-networks, ensuring that the generation of safety handling strategies and hierarchical self-healing action sequences always meets the requirements of multi-objective optimization.

[0246] In this embodiment, the decision generation and output module 35 generates a security handling strategy and a hierarchical self-healing action sequence adapted to the current threat through reinforcement learning, and transmits the security handling strategy and the hierarchical self-healing action sequence to the verification and optimization module 36. The specific implementation is as follows:

[0247] Reinforcement learning algorithm selection and network structure:

[0248] The decision generation and output module 35 uses a deep Q-network (DQN) as the core reinforcement learning algorithm, and constructs the reinforcement learning state space output by the state space construction module 32. As input, the effective motion space output by the motion space construction and filtering module 33. As the action selection range, the multi-objective adaptive reward and punishment function output by the reward and punishment function construction module 34 is used. As a basis for value assessment.

[0249] The structure of a deep Q-network is as follows:

[0250] Input layer: 3-dimensional, corresponding to the reinforcement learning state space. The three core dimensions ;

[0251] Hidden layers: 2 layers, 64 neurons per layer, using the ReLU activation function;

[0252] Output layer: Dimension equals effective action space The number of action combinations is calculated, and the Q-value (value assessment value) of each action combination is output. The effective action space is also considered. The action combinations in the process are deduplicated based on the unique identifier of "threat type-execution sequence", and duplicate handling-self-healing action combinations are eliminated to ensure that the dimension of the DQN output layer is the number of unique action combinations after deduplication.

[0253] Loss function and parameter updates:

[0254] Loss function of deep Q network The calculation formula is:

[0255] ;

[0256] in:

[0257] These represent the parameters used to evaluate the network.

[0258] This represents the expectation operation;

[0259] This represents the current state vector (i.e., the reinforcement learning state space). );

[0260] This indicates the selected action combination (from the effective action space). );

[0261] This represents the reward / penalty value obtained after performing an action (i.e., the multi-objective adaptive reward / penalty function). );

[0262] This represents the next state vector after the action is performed.

[0263] The target Q value is expressed by the following formula: ,in, This represents the discount factor, which can be 0.95, and is used to balance immediate rewards and long-term rewards. The parameters representing the target network are periodically copied and updated from the evaluation network; Indicates the next state The optimal action combination.

[0264] The decision generation and output module 35 iteratively updates and evaluates the network parameters using the gradient descent algorithm. until the loss function (Convergence threshold).

[0265] Decision result generation and output:

[0266] After convergence, the decision generation and output module 35 selects the action combination with the largest Q value and organizes it according to the execution sequence as follows:

[0267] Security response strategy: includes core information such as action type, execution node, and execution timing;

[0268] The hierarchical self-healing action sequence is as follows: Level 1 (core node repair), Level 2 (link reconstruction), and Level 3 (resource scheduling), clearly defining the execution sequence and priority of each action.

[0269] The decision generation and output module 35 transmits the generated safety handling strategy and hierarchical self-healing action sequence to the verification and optimization module 36 through the internal standardized data transmission interface. At the same time, it retains the decision log (including action combination identifier, Q value, reward and punishment value, and timing information) locally for subsequent system iteration optimization and data traceability.

[0270] In this embodiment, the verification and optimization module 36, based on the AI ​​agent's business capability baseline and security threat handling requirements, simultaneously verifies and iteratively optimizes the effectiveness of the security handling strategy and the business security of the hierarchical self-healing action sequence. After the verification passes, the decision generation and output module 35 outputs the security handling strategy and hierarchical self-healing action sequence to the threat handling and self-healing execution unit 4. The process by which the verification and optimization module 36 verifies and iteratively optimizes the security handling strategy and hierarchical self-healing action sequence based on the AI ​​agent's business capability baseline and security threat handling requirements includes the following steps:

[0271] S36.1 Establish a three-dimensional verification mechanism of "business security + handling effectiveness + resource rationality" and set verification standards. This provides clear and quantifiable criteria for verifying security handling strategies and hierarchical self-healing action sequences, ensuring that verification covers three core dimensions: business continuity, threat handling effectiveness, and system resource capacity. It avoids decision-making flaws caused by verification based on a single dimension. The specific implementation is as follows:

[0272] The verification and optimization module 36, based on the AI ​​agent's business capability baseline and security threat handling requirements, constructs a three-dimensional verification standard consisting of a business security threshold, a handling effectiveness threshold, and a resource rationality threshold. Its core expression is:

[0273] ;

[0274] in:

[0275] This represents the three-dimensional verification standard, which is a vector containing three core thresholds and serves as the core criterion for determining whether the verification result is qualified.

[0276] This represents the business security threshold, which is dimensionless and can take a value of 0.3. It corresponds to the maximum allowable deviation rate of the AI ​​agent's business capabilities, which is set based on the business capability baseline. For core business scenarios, it can be lowered to 0.2.

[0277] This represents the threshold for the effectiveness of the response. It is dimensionless and can be set to 0.8, corresponding to the minimum expected success rate of the security response strategy. It is set based on the security threat response requirements, and can be increased to 0.9 for high-risk threat scenarios.

[0278] This represents the threshold for resource rationality. It is dimensionless and can be set to 0.5. It corresponds to the maximum allowable proportion of resources reserved by the AI ​​agent. It is set based on the system resource configuration and can be lowered to 0.4 in resource-scarce scenarios.

[0279] Meanwhile, the verification optimization module 36 defines quantifiable verification metrics and calculation logic for each verification dimension, establishing a one-to-one correspondence with the thresholds:

[0280] Business security verification indicators: (Predicted deviation rate of business capabilities after execution of the graded self-healing action sequence), calculation logic and Consistent, obtained by fitting historical action-business impact data based on the security threat knowledge graph;

[0281] Effectiveness verification indicators for handling: (Predicted success rate of safety handling strategy), calculation logic and Consistent, obtained by fitting historical case data of security threat knowledge graphs;

[0282] Resource rationality verification indicators: (The percentage of resources consumed after the execution of the safety handling strategy and the graded self-healing action sequence) is calculated using the following formula:

[0283] ;

[0284] in:

[0285] Predicted resource consumption (CPU, memory, computing power) for executing the action sequence;

[0286] The total amount of reserved resources for AI agents;

[0287] and All of these can be retrieved from the security threat knowledge graph.

[0288] S36.2, The effectiveness of the synchronous verification security handling strategy and the business security of the hierarchical self-healing action sequence were not achieved. In standard cases, gradient descent is used to adjust reinforcement learning parameters. This enables precise optimization of the decision generation process, ensuring that each iteration addresses validation failures specifically. The specific implementation is as follows:

[0289] Synchronous verification execution:

[0290] The verification and optimization module 36 receives the initial security handling strategy and hierarchical self-healing action sequence transmitted by the decision generation and output module 35, based on... The corresponding three-dimensional verification metrics are used to perform synchronous verification operations. The verification judgment formula is as follows:

[0291] ;

[0292] in:

[0293] The verification result is dimensionless, with 1 indicating that the verification passed and 0 indicating that the verification failed.

[0294] This represents the predicted deviation rate of business capabilities, is dimensionless, and ranges from [0,1].

[0295] This represents the business security threshold, is dimensionless, and can take a value of 0.3.

[0296] This represents the predicted success rate of the treatment, which is dimensionless and ranges from [0,1].

[0297] This represents the threshold for the effectiveness of the treatment; it is dimensionless and can take a value of 0.8.

[0298] This represents the percentage of resource consumption; it is dimensionless and ranges from [0,1].

[0299] This represents the threshold for resource rationality; it is dimensionless and has a value of 0.5.

[0300] Reinforcement learning parameters Adjustment:

[0301] when At the same time, the verification optimization module 36 uses the gradient descent algorithm to adjust the parameters of the deep Q-network evaluation network in the decision generation and output module 35 for dimensions that fail verification. The formula is adjusted as follows:

[0302] ;

[0303] in:

[0304] This represents the adjusted reinforcement learning evaluation network parameters, which is the set of weights and biases for a deep Q-network;

[0305] This represents the network parameters before adjustment in the reinforcement learning evaluation, compared to... Consistent dimensions;

[0306] This represents the learning rate, which is dimensionless and takes a value of 0.001. It is used to control the step size of parameter adjustment to avoid excessive adjustment that could cause the model to fail to converge.

[0307] Represents the loss function The gradient reflects the direction and degree of the influence of parameter changes on the loss value, and the calculation logic is consistent with the gradient of the loss function in the decision generation and output module 35.

[0308] The specific rules for parameter adjustment are as follows:

[0309] like (Business security is not up to standard): Increase the weight of business capability-related indicators in the loss function and strengthen the optimization direction of business security in gradient calculation;

[0310] like (Inadequate handling effectiveness): Increase the weight of threat handling-related indicators in the loss function and strengthen the optimization direction of handling effectiveness in gradient calculation;

[0311] like (Resource rationality not up to standard): Add a penalty term for resource consumption to the loss function to strengthen the optimization direction of resource rationality in gradient calculation.

[0312] When two or more dimensions fail the validation simultaneously, parameter adjustments follow the scenario adaptation priority rule, as follows:

[0313] In core business scenarios, priority should be given to adjusting parameters corresponding to non-compliance with business security standards, and then optimization should be carried out in order of effectiveness of handling and rationality of resources.

[0314] In high-risk threat scenarios, priority should be given to adjusting parameters where the effectiveness of the response is not up to standard, and then optimization should be carried out in the order of business security and resource rationality.

[0315] In resource-constrained scenarios, priority should be given to adjusting parameters where resource rationality is not met, followed by optimization in the order of business security and handling effectiveness.

[0316] Furthermore, for multi-dimensional adjustments under the same priority, the relevant parameters of the loss function are updated synchronously using a weighted distribution method to ensure that gradient calculation can respond to multiple optimization directions simultaneously.

[0317] S36.3, Optimize the parameters The data is then sent back to the decision generation and output module 35 to regenerate the security handling strategy and the hierarchical self-healing action sequence until verification is passed. This forms a closed-loop iterative mechanism of "verification-optimization-regeneration," ensuring that the final output decision fully meets the baseline of the AI ​​agent's business capabilities and the requirements for handling security threats. The specific implementation is as follows:

[0318] The verification and optimization module 36 transmits the adjusted reinforcement learning parameters through the internal standardized parameter transmission interface. The data is sent back to the decision generation and output module 35, along with a marker for dimensions that failed the verification, so that the decision generation and output module 35 can accurately update the model parameters.

[0319] Decision generation and output module 35 receives Then, immediately update the parameters of the deep Q-network evaluation network, and based on the updated model, reconstruct the output of module 32 in state space. The output of the input, action space construction and filtering module 33 The output of module 34, which constructs the action range and reward / penalty functions. Based on the value, a new security handling strategy and a hierarchical self-healing action sequence are generated and transmitted again to the verification and optimization module 36.

[0320] The verification optimization module 36 repeats the synchronous verification operation of S36.2 on the newly generated decision result. If the verification result... If, then terminate the iteration; if Then, the parameter adjustment and feedback operation will be performed again.

[0321] Meanwhile, to avoid infinite iteration, the verification optimization module 36 sets a maximum iteration threshold. The value is 5; if the number of iterations reaches... If the verification still fails, the verification optimization module 36 will trigger an alarm mechanism, report the abnormal information to the collaborative management and iterative optimization unit 5, and output the current optimal decision result as an alternative.

[0322] Furthermore, once the verification is successful, the verification optimization module 36 sends a "verification successful" instruction to the decision generation and output module 35. The decision generation and output module 35 then outputs the final security handling strategy and hierarchical self-healing action sequence to the threat handling and self-healing execution unit 4 through a standardized data transmission interface. At the same time, it retains a complete verification iteration log locally (including parameter adjustment values, verification index values, and iteration counts for each iteration) for subsequent system iteration optimization and data traceability.

[0323] It should be added that the core transmission triggering conditions for the above-mentioned decision results to the threat handling and self-healing execution unit 4 are divided into three categories: First, when the verification passes on the first try, the transmission is triggered immediately; second, when the iteration reaches the maximum number of times and still fails, the transmission of alternative solutions is triggered and an alarm is simultaneously sent to the collaborative management and iterative optimization unit 5; third, when the input data of the security situation awareness and threat identification unit 2 is updated, the decision is regenerated and the new decision results are transmitted synchronously.

[0324] Threat handling and self-healing execution unit 4 receives the verified security handling strategy and self-healing action sequence output by the handling-self-healing decision generation unit 3, executes security threat handling operations such as attack blocking and malicious code removal, and simultaneously executes the AI ​​agent's hierarchical self-healing operations such as configuration restoration and node repair. During execution, it verifies the business capability status against the AI ​​agent's business capability baseline in real time, collects handling result data, self-healing result data, and business capability status data, forms full-process execution feedback data, and outputs it to the collaborative management and iterative optimization unit 5. Threat handling and self-healing execution unit 4 includes a decision instruction receiving module 41, a threat handling execution module 42, a hierarchical self-healing execution module 43, a real-time business capability verification module 44, and a data collection and feedback module 45, wherein: 0

[0325] In this embodiment, the decision instruction receiving module 41 receives the verified security handling strategy and hierarchical self-healing action sequence output by the handling-self-healing decision generation unit 3, realizing accurate reception, format parsing and compliance verification of decision data, and providing a standardized execution basis for the threat handling execution module 42. The specific implementation is as follows:

[0326] Deploy an encrypted standardized direct connection interface to receive only verified decision data; parse the security handling strategy into four structured elements: action type, execution target, execution sequence, and operation parameters; decompose the hierarchical self-healing action sequence into instructions with unique execution identifiers according to hierarchical rules; after completing the data field integrity verification, push the structured data to the threat handling execution module 42 and the hierarchical self-healing execution module 43 respectively, while retaining a local copy of the data for execution traceability.

[0327] In this embodiment, the threat handling execution module 42 performs security threat handling operations such as attack blocking and malicious code removal according to the security handling strategy, so as to achieve precise blocking and removal of security threats and curb the spread of threats from the source. The specific implementation is as follows:

[0328] Based on the action type and execution goal of the handling strategy, match the corresponding security execution engines such as network firewalls and malware removal; for attack blocking, issue commands for network isolation, attack traffic interception, and dangerous port blocking, and monitor the blocking effect in real time; for malware removal, perform a comprehensive scan of infected nodes, perform malware isolation and removal operations, and complete residual detection; add an execution status identifier to each handling operation, and synchronously record core information such as execution time, execution goal, and operation result.

[0329] In this embodiment, the hierarchical self-healing execution module 43 synchronously executes the configuration restoration and node repair hierarchical self-healing operations of the AI ​​agent according to the hierarchical self-healing action sequence, and coordinates with the threat handling operation to quickly restore the normal operation state of the AI ​​agent. The specific implementation is as follows:

[0330] Strictly follow the hierarchical priority, execution sequence and dependencies of the self-healing action sequence, and trigger operations synchronously with the threat handling execution module 42; for configuration restoration, retrieve normal templates from the configuration baseline library of the security threat knowledge graph construction unit 1, complete the configuration restoration and verify its effectiveness;

[0331] Meanwhile, for node repair, based on the priority of primary core nodes and secondary ordinary nodes, hot repair and cold repair modes are used to complete operations such as process restart and reinstallation of damaged components. If the preceding actions are successfully executed, the subsequent actions are triggered. If the execution fails, the backup self-healing plan is immediately started, and the full details of the self-healing operation are recorded.

[0332] In this embodiment, the real-time business capability verification module 44 verifies the business capability status of the AI ​​agent by comparing it with the AI ​​agent's business capability baseline in real time during the handling and self-healing operation process. This avoids excessive impact on the business and ensures business continuity. The specific implementation is as follows:

[0333] The system retrieves baseline data on the business capabilities of AI agents from security threat knowledge graph construction unit 1 to establish a local database, and collects core business indicators such as business response time and task completion rate at a sampling frequency of 1 second / time. The system then uses the calculation logic of state space construction module 32 to obtain the business capability deviation rate. If the deviation rate exceeds the threshold Immediately issue an operation pause / adjustment command to the downstream execution module, and resume execution after the business capability recovers to the threshold range; record the sampling time, indicator value, deviation rate and operation intervention in real time to form a complete verification log.

[0334] In this embodiment, the data acquisition and feedback module 45 collects processing result data, self-healing result data, and business capability status data to form full-process execution feedback data. This full-process execution feedback data is then output to the collaborative management and iterative optimization unit 5, providing real and complete execution data support for system iterative optimization. The specific implementation is as follows:

[0335] The core data of handling results, self-healing results, and business capability status are collected from the threat handling execution module 42, the hierarchical self-healing execution module 43, and the real-time business capability verification module 44, respectively.

[0336] At the same time, the collected data is structured and integrated, invalid data is cleaned, and the data is classified and labeled according to the dimensions of execution batch, operation type, and execution node.

[0337] Finally, the processed full-process execution feedback data is output to the collaborative management and iterative optimization unit 5 through an encrypted transmission interface, and local persistent storage is completed to support data traceability and review.

[0338] The Collaborative Control and Iterative Optimization Unit 5 receives full-process execution feedback data from the Threat Handling and Self-Healing Execution Unit 4. Based on the security threat knowledge graph output by the Security Threat Knowledge Graph Construction Unit 1, it implements threat intelligence synchronization among multiple AI agents, coordinated action linkage, and unified scheduling of self-healing resources. It iteratively optimizes the reinforcement learning-based decision generation process of the Handling-Self-Healing Decision Generation Unit 3 based on the execution feedback data, updates the entity and relationship data of the security threat knowledge graph, and feeds the updated security threat knowledge graph data back to the Security Threat Knowledge Graph Construction Unit 1. The Collaborative Control and Iterative Optimization Unit 5 includes a dual-source data receiving module 51, a collaborative control module 52, a decision iterative optimization module 53, a knowledge graph update module 54, and a data feedback module 55, wherein:

[0339] In this embodiment, the dual-source data receiving module 51 receives the full-process execution feedback data output by the threat handling and self-healing execution unit 4, as well as the security threat knowledge graph data output by the security threat knowledge graph construction unit 1. This completes the accurate reception, format unification, and validity verification of the dual-source data, providing a standardized and highly reliable unified data foundation for subsequent collaborative management, decision optimization, and graph updates. The specific implementation is as follows:

[0340] A dual-channel encrypted standardized data interface is deployed, directly connected to the data acquisition and feedback module 45 and the knowledge graph generation and output module 14 respectively, to receive full-process execution feedback data and security threat knowledge graph data. The above two types of data are parsed and structured, and the data identification fields and transmission formats are unified. Completeness and timeliness are checked, and invalid data with missing core information or exceeding the time limit are removed. The dual-source data that passes the check are integrated and labeled according to the dimensions of "threat handling - self-healing execution - knowledge graph", and after being lightly cached locally, they are pushed to the collaborative management module 52, the decision iteration optimization module 53, and the knowledge graph update module 54 respectively.

[0341] In this embodiment, the collaborative management module 52, based on security threat knowledge graph data, implements threat intelligence synchronization, coordinated action, and unified scheduling of self-healing resources among multiple AI agents, thereby achieving global security collaborative protection among multiple AI agents and avoiding the limitations of single agent protection. The specific implementation is as follows:

[0342] A collaborative management and control platform for multiple AI agents is built based on the entity relationships of a security threat knowledge graph.

[0343] Threat intelligence synchronization: Establish a multi-agent shared threat intelligence database, synchronize threat types, impact range, and handling methods of each agent in real time, and push early warning information to agents with similar vulnerabilities based on graph correlations;

[0344] Coordinated action response: Establish rules for coordinated action response across agents to avoid duplicate or conflicting responses, and trigger coordinated blocking, isolation, and other response actions in a unified manner for threats propagating across agents;

[0345] Unified scheduling of self-healing resources: Integrate the self-healing resources (computing power, repair templates, and backup nodes) of each AI agent to establish a global resource pool. Based on the damage level and business priority of each agent, dynamically allocate self-healing resources according to graph data, and prioritize the repair of agent nodes corresponding to core businesses.

[0346] In this embodiment, the decision iteration optimization module 53 iteratively optimizes the decision generation process of the handling-self-healing decision generation unit 3 based on reinforcement learning according to the full-process execution feedback data, thereby improving the accuracy, adaptability, and business friendliness of the decision strategy. The specific implementation is as follows:

[0347] Key performance indicators (KPIs) for decision-making execution are extracted from the feedback data throughout the entire process, including threat handling success rate, business capability recovery rate, business interruption duration, and resource consumption ratio. The actual values ​​of these KPIs are compared with preset optimization targets to analyze shortcomings in the decision generation process, such as poor adaptability of reward / penalty function weights and unreasonable action space filtering rules. The core parameters and rules of the handling-self-healing decision generation unit 3 are then iteratively optimized, including adjusting the weight coefficients of the reward / penalty function, optimizing the quantification logic of the state space dimension, improving the irreversible damage filtering rules of the action space, and updating the model parameters of the deep Q-network. The optimized solutions are then pushed to the corresponding modules of the handling-self-healing decision generation unit 3 to complete the iterative update of the decision generation process.

[0348] In this embodiment, the knowledge graph update module 54 updates the entity and relationship data of the security threat knowledge graph based on the full-process execution feedback data, enriches the knowledge dimensions of the graph, and improves the adaptability and accuracy of the graph to AI intelligent agent security scenarios. The specific implementation is as follows:

[0349] Extract graph update information from the feedback data of the entire process execution, including new security threat / vulnerability entities, changes in the attributes of existing entities (such as threat handling effectiveness, vulnerability remediation rate), and additions or corrections to the relationships between entities (such as adding a relationship where a threat exploits a vulnerability, or correcting the relationship range between a threat and affected nodes).

[0350] Standardize the naming and attribute labeling of newly added entities and supplement them to the security threat knowledge graph;

[0351] Dynamically update existing entity attributes and relationships between entities;

[0352] Ontology reasoning tools are used to verify the consistency and integrity of the updated knowledge graph, eliminating conflicting and invalid updates to ensure the accuracy of the graph data.

[0353] In this embodiment, the data feedback module 55 feeds the updated security threat knowledge graph data back to the security threat knowledge graph construction unit 1, realizing the dynamic iteration of the security threat knowledge graph and providing updated knowledge support for the system's subsequent security situation awareness, threat identification, and response-self-healing decision generation. The specific implementation is as follows:

[0354] The verified updated security threat knowledge graph data is standardized and packaged according to the receiving format of the security threat knowledge graph construction unit 1, including newly added entity data, attribute update data, relationship correction data, and update logs; it is then fed back to the graph generation output module 14 of the security threat knowledge graph construction unit 1 through an encrypted standardized data interface; the graph update log is pushed synchronously, specifying the update content, update basis, and update time, to ensure that the security threat knowledge graph construction unit 1 completes the unified update, persistent storage, and system-wide synchronization of the graph.

[0355] Those skilled in the art will understand that the process of implementing all or part of the steps of the above embodiments can be carried out by hardware or by a program instructing the relevant hardware.

[0356] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. An integrated AI intelligent agent security threat handling and self-healing collaborative system that combines knowledge graphs and reinforcement learning, characterized in that: include: The security threat knowledge graph construction unit (1) collects AI agent operation data, security alarm data, vulnerability detection data, and AI agent business capability baseline data. Through named entity recognition, entity relationship alignment, and ontology construction technology, it constructs a security threat knowledge graph containing core entities and related relationships of agent nodes, business links, security threats, vulnerabilities, and business capability baselines, and outputs the security threat knowledge graph data to the security situation awareness and threat identification unit (2). The security situation awareness and threat identification unit (2) receives the security threat knowledge graph data output by the security threat knowledge graph construction unit (1), performs entity relationship reasoning based on the entity relationship in the security threat knowledge graph data, completes the real-time security situation awareness and threat identification of the AI ​​agent, generates and outputs standardized threat relationship data, AI agent business capability baseline data, business link and security status relationship data, and transmits them synchronously to the handling-self-healing decision generation unit (3). The disposal-self-healing decision generation unit (3) receives standardized threat association data, AI agent business capability baseline data, and business link and security status association data output by the security situation awareness and threat identification unit (2); uses the security threat knowledge graph as a prior constraint for reinforcement learning, and constructs a reinforcement learning state space based on the threat status, vulnerability status, and degree of business capability impairment; constructs an action space coupled with disposal actions and self-healing actions based on the business link and security status association rules, and eliminates actions that will cause irreversible damage to the business capabilities of the AI ​​agent; constructs a multi-objective reward and punishment function with the threat disposal success rate, business capability recovery rate, system self-healing success rate, and business interruption duration as optimization objectives, and generates a security disposal strategy and graded self-healing action sequence adapted to the current threat through reinforcement learning; performs business capability impact verification on the action sequence based on the business capability baseline, and performs iterative optimization if the preset verification threshold is not reached, and outputs the verified security disposal strategy and self-healing action sequence to the threat disposal and self-healing execution unit (4). Threat handling and self-healing execution unit (4) receives the verified security handling strategy and self-healing action sequence output by the handling-self-healing decision generation unit (3), performs security threat handling operations such as attack blocking and malicious code removal, and simultaneously performs AI agent hierarchical self-healing operations such as configuration restoration and node repair. During the execution process, the business capability status is verified in real time by comparing with the AI ​​agent business capability baseline, and the handling result data, self-healing result data, and business capability status data are collected to form full-process execution feedback data, which is then output to the collaborative management and iterative optimization unit (5). The collaborative management and iterative optimization unit (5) receives the full-process execution feedback data output by the threat handling and self-healing execution unit (4), and implements threat intelligence synchronization, collaborative action linkage, and unified scheduling of self-healing resources among multiple AI agents based on the security threat knowledge graph output by the security threat knowledge graph construction unit (1). The iterative optimization of the handling-self-healing decision generation unit (3) based on the reinforcement learning decision generation process according to the execution feedback data updates the entity and relationship data of the security threat knowledge graph, and feeds back the updated security threat knowledge graph data to the security threat knowledge graph construction unit (1).

2. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 1, characterized in that, The security threat knowledge graph construction unit (1) includes a data acquisition module (11), a data association processing module (12), an ontology construction module (13), and a graph generation and output module (14), wherein: The data acquisition module (11) collects AI agent operation data, security alarm data, vulnerability detection data, and AI agent business capability baseline data; The data association processing module (12) performs named entity recognition and entity relationship alignment on the data collected by the data acquisition module (11); The ontology construction module (13) constructs the core entities and related relationships of intelligent agent nodes, business links, security threats, vulnerabilities, and business capability baselines based on the processing results of the data association processing module (12). The graph generation and output module (14) generates a security threat knowledge graph based on the construction results of the ontology construction module (13) and outputs the security threat knowledge graph data to the security situation awareness and threat identification unit (2).

3. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 2, characterized in that, The security situation awareness and threat identification unit (2) includes a data receiving and parsing module (21), an entity association reasoning module (22), a situation awareness and threat identification module (23), and a data generation and output module (24), wherein: The data receiving and parsing module (21) receives the security threat knowledge graph data output by the security threat knowledge graph construction unit (1), and performs format parsing and validity verification on the security threat knowledge graph data; The entity association reasoning module (22) performs entity association reasoning based on the entity associations in the security threat knowledge graph data parsed by the data receiving and parsing module (21), and outputs the entity association reasoning results. The situational awareness and threat identification module (23) receives the entity association reasoning results output by the entity association reasoning module (22) and completes the real-time security situational awareness and threat identification of the AI ​​agent. The data generation and output module (24) generates and outputs standardized threat-related data, AI agent business capability baseline data, business link and security status-related data based on the identification results of the situation awareness and threat identification module (23), and transmits them synchronously to the handling-self-healing decision generation unit (3).

4. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 3, characterized in that, The disposal-self-healing decision generation unit (3) includes a data receiving module (31), a state space construction module (32), an action space construction and filtering module (33), a reward and punishment function construction module (34), a decision generation and output module (35), and a verification and optimization module (36), wherein: the data receiving module (31) receives standardized threat correlation data, AI intelligent agent business capability baseline data, and business link and security status correlation data output by the security situation awareness and threat identification unit (2); The state space construction module (32) constructs a reinforcement learning state space using the security threat knowledge graph as a prior constraint for reinforcement learning. The action space construction and filtering module (33) constructs and filters the action space coupled with handling actions and self-healing actions based on the association rules between business links and security status; the reward and punishment function construction module (34) constructs a multi-objective adaptive reward and punishment function with a preset optimization goal. The decision generation and output module (35) generates a security handling strategy and a hierarchical self-healing action sequence adapted to the current threat through reinforcement learning, and transmits the security handling strategy and hierarchical self-healing action sequence to the verification and optimization module (36). The verification and optimization module (36) verifies and iteratively optimizes the effectiveness of the security handling strategy and the business security of the hierarchical self-healing action sequence in accordance with the baseline of AI intelligent agent business capabilities and security threat handling requirements. After the verification is passed, the decision generation and output module (35) outputs the security handling strategy and hierarchical self-healing action sequence to the threat handling and self-healing execution unit (4).

5. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 4, characterized in that, The state space construction module (32) uses the security threat knowledge graph as a prior constraint for reinforcement learning to construct the reinforcement learning state space. The process includes the following steps: S32.

1. Based on the entity relationships in the security threat knowledge graph, determine the three core state dimensions: threat, vulnerability, and business impairment. S32.2 Constructing a reinforcement learning state space based on three core state dimensions. ; S32.3, Strengthen the learning state space The output is sent to the decision generation and output module (35) as the state input for reinforcement learning.

6. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 5, characterized in that, The process by which the action space construction and filtering module (33) constructs and filters the action space coupled with handling actions and self-healing actions based on the business link and security status association rules includes the following steps: S33.1 Establish the linkage and coupling logic between treatment actions and self-healing actions, and clarify the timing and correlation of action combinations; S33.2 Construct a treatment-self-healing coupled action space, introduce an irreversible damage screening mechanism for business capabilities, and obtain an effective action space. S33.3, Effective motion space The output is sent to the decision generation and output module (35) as the range of reinforcement learning action selection.

7. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 6, characterized in that, The reward and punishment function construction module (34) constructs a multi-objective adaptive reward and punishment function based on a preset optimization objective. The process includes the following steps: S34.1, the four core objectives are: success rate of integrated threat handling, business capability recovery rate, system self-healing success rate, and business interruption duration. S34.2 Constructing a multi-objective adaptive reward and punishment function , by weight allocation, the optimization focus of different threat scenarios is realized, and a negative penalty mechanism is introduced; S34.3, the multi-objective adaptive reward and punishment function is used as the value evaluation basis of the decision generation and output module (35) reinforcement learning, and guides the iterative generation of security handling strategies and hierarchical self-healing action sequences.

8. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 7, characterized in that, The verification and optimization module (36) performs the following steps to verify and iteratively optimize the security handling strategy and the hierarchical self-healing action sequence based on the AI ​​agent's business capability baseline and security threat handling requirements: S36.1 Establish a three-dimensional verification mechanism of "business security + handling effectiveness + resource rationality" and set verification standards. S36.2, The effectiveness of the synchronous verification security handling strategy and the business security of the hierarchical self-healing action sequence were not achieved. In standard cases, gradient descent is used to adjust reinforcement learning parameters. S36.3, Optimize the parameters The data is sent back to the decision generation and output module (35) to regenerate the safety handling strategy and the hierarchical self-healing action sequence until the verification is passed.

9. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 8, characterized in that, The threat handling and self-healing execution unit (4) includes a decision instruction receiving module (41), a threat handling execution module (42), a hierarchical self-healing execution module (43), a real-time business capability verification module (44), and a data acquisition and feedback module (45), wherein: The decision instruction receiving module (41) receives the verified safety handling strategy and graded self-healing action sequence output by the handling-self-healing decision generation unit (3); The threat handling execution module (42) performs security threat handling operations such as attack blocking and malicious code removal according to the security handling strategy; The hierarchical self-healing execution module (43) synchronously executes the configuration restoration and node repair of the AI ​​agent hierarchical self-healing operation according to the hierarchical self-healing action sequence. The real-time business capability verification module (44) verifies the business capability status of the AI ​​agent by comparing it with the AI ​​agent business capability baseline in real time during the handling and self-healing operation process. The data acquisition and feedback module (45) collects the handling result data, self-healing result data, and business capability status data to form full-process execution feedback data, and outputs the full-process execution feedback data to the collaborative management and iterative optimization unit (5).

10. The AI ​​intelligent agent security threat handling-self-healing collaborative integrated system integrating knowledge graph and reinforcement learning as described in claim 9, characterized in that, The collaborative management and iterative optimization unit (5) includes a dual-source data receiving module (51), a collaborative management module (52), a decision iterative optimization module (53), a knowledge graph update module (54), and a data feedback module (55), wherein: The dual-source data receiving module (51) receives the full-process execution feedback data output by the threat handling and self-healing execution unit (4), and the security threat knowledge graph data output by the security threat knowledge graph construction unit (1); The collaborative management module (52) implements threat intelligence synchronization, coordinated action linkage and unified scheduling of self-healing resources among multiple AI agents based on security threat knowledge graph data. The decision iteration optimization module (53) iteratively optimizes the decision generation process of the treatment-self-healing decision generation unit (3) based on reinforcement learning according to the full-process execution feedback data; The knowledge graph update module (54) updates the entity and relationship data of the security threat knowledge graph based on the full-process execution feedback data; The data feedback module (55) feeds back the updated security threat knowledge graph data to the security threat knowledge graph construction unit (1).

Images